diff --git a/CHANGELOG.md b/CHANGELOG.md index 3fe370e..53c22c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,34 +10,34 @@ AL123456.3 GenBank record at exactly its coordinates, strand and name: 4031 of directly comparable with 1.0.1. ### ๐Ÿ”ฌ Correctness -- **Paralog copies no longer take another gene's place**: a protein with a paralog was placed at both loci, and which copy survived came down to insertion order. On H37Rv itself that lost Rv1150, Rv1945 and Rv2828c; on M. bovis it reported genes the strain lacks (the mce3 operon of RD7, the phiRv2 prophage of RD11, plcBC of RD5) as copies sitting on their paralogs. Each placement is now checked against where its protein belongs, predicted from the sample's own offset to the reference so that indels do not throw it off +- **Paralog copies no longer take another gene's place**: a protein with a paralog was placed at both loci, and which copy survived came down to insertion order. On H37Rv itself that lost Rv1150, Rv1945 and Rv2828c; on *M. bovis* it reported genes the strain lacks (the mce3 operon of RD7, the phiRv2 prophage of RD11, plcBC of RD5) as copies sitting on their paralogs. Each placement is now checked against where its protein belongs, predicted from the sample's own offset to the reference so that indels do not throw it off - **Strand of a resolved conflict**: taken from the protein's first placement anywhere in the genome instead of the one being resolved, so esxM (Rv1792), Rv3466 and two IS6110 copies came out at their own coordinates on the wrong strand; a Bakta call chosen over an alignment also took the alignment's strand - **Reference pseudogenes keep their extent**: PE21 (Rv2099c) and PPE48 (Rv3022c), which the reference annotates without a stop, were extended to the next one, and oxyR' (Rv2427A) and Rv3216 lost the partial final codon the reference gives them - **The 1.0.1 fix for reference pseudogenes never fired**: it looked for reference coordinates in a column that has not carried them since March 2026, and its tests wrote them in by hand. `check_cds.py` now reads the reference proteome instead - **Alignments that stop short are completed**: miniprot can miss a protein's N-terminus (Rv3611 on CP110619) or leave a repetitive C-terminus unaligned further from its stop than the 300 bp searched; the reference protein now says how much is missing -- **Ties between copies of a gene** go to the one Bakta also called, then to the one where the protein belongs by synteny, not the one that comes first in the genome (E. coli K-12 b2635) +- **Ties between copies of a gene** go to the one Bakta also called, then to the one where the protein belongs by synteny, not the one that comes first in the genome (*E. coli* K-12 b2635) - **Draft assemblies**: genes were compared on coordinates alone, and every contig starts at 1, so cut into 40 contigs CDC1551 had 87% of its proteins translated from the wrong contig. Every gene is now matched, translated and written on its own contig, and Bakta runs with `--keep-contig-headers` so that its contig names match the FASTA's - **Case 4 is grouped strand by strand**: about 8% of its groups (101 of 1255 on H37Rv) put an alignment on one strand together with a Bakta call on the other - **A complete alignment is not displaced by a longer call in its frame**: such a call ends at the same stop and only starts further upstream (plcD on CDC1551). Nor does a call that spans several reference genes win against them (Rv3022c and Rv3021c, one reading frame in CDC1551) - **Reference genes are protected again**: the check looked for coordinates in identifiers that no longer carry them, so any call of similar length could displace a reference gene (Rv1614) -- **Missing genes are recovered by what covers them**: genes stored under Bakta locus tags looked missing by name, so a placement a codon off its gene went back in as a second copy. With the case 4 changes, ORFs annotated twice fall from 10 to 3 on CDC1551 and from 16 to 6 on M. bovis, where the reference has none -- **Real copies of a gene are kept**: `deduplicate_rvs.py` deleted every copy of a gene but one, 34 genes on E. coli O157:H7 Sakai, which were then missing from the annotation and from the duplication analysis. Further copies get identifiers of their own -- **Genes that lost their stop no longer swallow the next gene**: `check_cds.py` extended a gene without a stop to the next one in its frame, which after an indel is the next gene's, so that gene was annotated a second time in the same frame (M. bovis Rv0590 over Rv0590A, Rv3373 over Rv3374, Rv2293c over Rv2292c). Such a gene now keeps its extent as a pseudogene, unless the extension only restores residues its reference protein has. ORFs annotated twice fall from 3 to 2 on CDC1551 and from 6 to 3 on M. bovis, and none of those left comes from an extension -- **Small proteins**: miniprot placed none of the 33 proteins of 8 to 29 residues in E. coli K-12, and put the toxin IbsC (b4665) only on its near-identical neighbour IbsD. Every protein without an exact placement is now looked for in the six-frame translation of the genome, which takes K-12 against itself from 4252 to all 4286 of its genes, its alternative-start isoforms counted with their gene, and Sakai from 198 to 220 of 220 -- **Frameshifts and lost stops**: a gene was extended to the next stop in its frame however far that was, and came out as a complete gene with a C-terminus its reference lacks; 18 in CDC1551 and 20 in M. bovis. It now keeps its extent and is flagged as a pseudogene when that stop lies more than ten residues past the end of the reference protein, as PGAP and Bakta treat a frameshift, which also covers a frame that runs into the next gene +- **Missing genes are recovered by what covers them**: genes stored under Bakta locus tags looked missing by name, so a placement a codon off its gene went back in as a second copy. With the case 4 changes, ORFs annotated twice fall from 10 to 3 on CDC1551 and from 16 to 6 on *M. bovis*, where the reference has none +- **Real copies of a gene are kept**: `deduplicate_rvs.py` deleted every copy of a gene but one, 34 genes on *E. coli* O157:H7 Sakai, which were then missing from the annotation and from the duplication analysis. Further copies get identifiers of their own +- **Genes that lost their stop no longer swallow the next gene**: `check_cds.py` extended a gene without a stop to the next one in its frame, which after an indel is the next gene's, so that gene was annotated a second time in the same frame (*M. bovis* Rv0590 over Rv0590A, Rv3373 over Rv3374, Rv2293c over Rv2292c). Such a gene now keeps its extent as a pseudogene, unless the extension only restores residues its reference protein has. ORFs annotated twice fall from 3 to 2 on CDC1551 and from 6 to 3 on *M. bovis*, and none of those left comes from an extension +- **Small proteins**: miniprot placed none of the 33 proteins of 8 to 29 residues in *E. coli* K-12, and put the toxin IbsC (b4665) only on its near-identical neighbour IbsD. Every protein without an exact placement is now looked for in the six-frame translation of the genome, which takes K-12 against itself from 4252 to all 4286 of its genes, its alternative-start isoforms counted with their gene, and Sakai from 198 to 220 of 220 +- **Frameshifts and lost stops**: a gene was extended to the next stop in its frame however far that was, and came out as a complete gene with a C-terminus its reference lacks; 18 in CDC1551 and 20 in *M. bovis*. It now keeps its extent and is flagged as a pseudogene when that stop lies more than ten residues past the end of the reference protein, as PGAP and Bakta treat a frameshift, which also covers a frame that runs into the next gene - **Genes a contig end cuts**: in a draft assembly they came out as pseudogenes, or as complete genes, without a word. A gene whose frame runs off the contig, or that opens at its edge without a start codon, is now marked partial at that end. In CDC1551 cut into 40 contigs, the larger piece of each of the 25 genes a cut runs through is marked so - **The other pieces of a cut gene**: miniprot reports only the placements within 5% of a protein's best, so only the larger piece of a gene a contig end cuts was annotated. What that placement lacks is now aligned again as a protein of its own, and its placement at the end of another contig is annotated under the gene's name and marked partial: 47 of the 50 pieces in the CDC1551 draft instead of 25, the other three being five or six codons long. A gene that opens at a contig's edge with a codon that can also start one, as one in six codons inside H37Rv genes can, is marked partial too when its protein begins inside its reference protein - **Disrupted copies of a gene**: deduplication kept further copies only when intact, so a whole copy that is a pseudogene was deleted with the pieces of split genes; it now stays when nearly as long as the reference protein -- **GFF3 references**: each CDS line was translated on its own, so a CDS of several lines came out in fragments, and proteins under 11 residues were dropped to hide them. The lines are joined per CDS and translated with the bacterial code, reading `transl_except`, and E. coli K-12's GFF3 gives the same 4318 proteins as its GenBank record, 24 of which were missing -- **Genes longer than their reference keep its name**: a Bakta call that won its locus over a shorter alignment in its frame kept no name, and the protein was missing from the table: Rv2024c in CDC1551 and M. bovis, mmpL6 (Rv1557, whole in M. bovis where TbD1 cut it in H37Rv), Rv1068c, Rv1917c and Rv3135 in M. bovis, fliD (b1923) and b0282 in Sakai. The call now takes the name when the alignment spans at least half of the protein, so a remnant does not name the gene it ends (the 67 residues of cyp141 that RD12 leaves in M. bovis) +- **GFF3 references**: each CDS line was translated on its own, so a CDS of several lines came out in fragments, and proteins under 11 residues were dropped to hide them. The lines are joined per CDS and translated with the bacterial code, reading `transl_except`, and *E. coli* K-12's GFF3 gives the same 4318 proteins as its GenBank record, 24 of which were missing +- **Genes longer than their reference keep its name**: a Bakta call that won its locus over a shorter alignment in its frame kept no name, and the protein was missing from the table: Rv2024c in CDC1551 and *M. bovis*, mmpL6 (Rv1557, whole in *M. bovis* where TbD1 cut it in H37Rv), Rv1068c, Rv1917c and Rv3135 in *M. bovis*, fliD (b1923) and b0282 in Sakai. The call now takes the name when the alignment spans at least half of the protein, so a remnant does not name the gene it ends (the 67 residues of cyp141 that RD12 leaves in *M. bovis*) - **Small proteins with a few substitutions**: only exact copies were looked for, so Sakai's ten K-12 micro-proteins with one to four residues changed were not annotated. A protein of up to 50 residues with no exact copy is now looked for with up to a fifth of its residues changed; a stop codon in place of a residue counts, and the gene is then a pseudogene (b1229) -- **Synteny no longer hangs on the order of the rows**: the anchors were sorted by start and searched by midpoint, so two anchors starting together sent a placement to whichever window the row order gave, and adding unrelated rows moved Sakai's b2127 24 kb and handed its locus to the paralog b1162. A locus several proteins are placed at exactly no longer anchors either: the single IS6110 of M. bovis made 34 anchors with reference positions all over the genome, and came out under all 17 IS6110 names of each of its genes; they are now Rv2814c and Rv2815c, its own +- **Synteny no longer hangs on the order of the rows**: the anchors were sorted by start and searched by midpoint, so two anchors starting together sent a placement to whichever window the row order gave, and adding unrelated rows moved Sakai's b2127 24 kb and handed its locus to the paralog b1162. A locus several proteins are placed at exactly no longer anchors either: the single IS6110 of *M. bovis* made 34 anchors with reference positions all over the genome, and came out under all 17 IS6110 names of each of its genes; they are now Rv2814c and Rv2815c, its own - **Genes without a COG category are counted once**: `count_annot.py` filled in a missing category as `Unassigned` and split it into letters, as it splits `JK` into J and K, so every gene eggNOG gave no category was counted once in A, D, E, G, I and U and twice in N and S, in `*-AMAP.COG.tsv` and in the report's COG figures and statistics, where it also shrank every other category's share. A test-profile run with a stand-in for eggNOG that gave none of its 4036 genes a category reported 4036 genes in each of those six and 8072 in N and S. Such a gene now counts once, as `NaN`, which the report shows as None, and its percentage is of all the genome's genes like every other category's - **COG categories R, W, X, Y and Z in the report**: its COG figures took their categories from a list that ran from A to V without R, so a gene eggNOG put in R, W, X, Y or Z was plotted as NA in the overview and in none of the per-category figures, which stayed empty for those five, X - Mobilome (prophages and transposons) among them. The categories now come from the report's own list of figures, A to Z and None, so each is plotted in its own. The statistics table never used that list and is unchanged -- **The QC table counts the pseudogenes AMAP called**: `annotation_qc.py` read the first of `Is_Pseudogene`, `pseudogene` and `is_pseudogene` that the master table has, and every master table has Mycobrowser's `Is_Pseudogene`, empty when Mycobrowser is off. So `pseudogene_count` in `*-AMAP.qc.tsv` and the report's QC table was 0 in every organism-agnostic run (E. coli Sakai against K-12, where AMAP called 129 pseudogenes), and in an MTBC run the number of the sample's genes named after one of the 13 that Mycobrowser marks as pseudogenes in H37Rv, whatever AMAP made of them: 13 for CDC1551, where AMAP calls 123 and finds 6 of those 13 intact. It now counts AMAP's own call, the master table's `pseudogene` column, the same genes the GFF3 marks `pseudo=true` and the GenBank `/pseudo` +- **The QC table counts the pseudogenes AMAP called**: `annotation_qc.py` read the first of `Is_Pseudogene`, `pseudogene` and `is_pseudogene` that the master table has, and every master table has Mycobrowser's `Is_Pseudogene`, empty when Mycobrowser is off. So `pseudogene_count` in `*-AMAP.qc.tsv` and the report's QC table was 0 in every organism-agnostic run (*E. coli* Sakai against K-12, where AMAP called 129 pseudogenes), and in an MTBC run the number of the sample's genes named after one of the 13 that Mycobrowser marks as pseudogenes in H37Rv, whatever AMAP made of them: 13 for CDC1551, where AMAP calls 123 and finds 6 of those 13 intact. It now counts AMAP's own call, the master table's `pseudogene` column, the same genes the GFF3 marks `pseudo=true` and the GenBank `/pseudo` - **The report's pseudogene chart counts AMAP's calls, sample by sample**: its Pseudogene Detection section counted the master table's genes flagged in Mycobrowser's `Is_Pseudogene` or in AMAP's `pseudogene` column, lineage by lineage, as "Pseudogenes per Lineage (Mycobrowser annotation)". `Is_Pseudogene` says which genes are pseudogenes in H37Rv, not in the sample, so CDC1551 came out at 129 where AMAP calls 123 and the QC table says 123, the other 6 being genes Mycobrowser marks that CDC1551 has intact (Rv0947c, Rv1150, Rv2023A, Rv2098c, Rv3021c, Rv3324A), and without Mycobrowser the chart showed AMAP's calls under Mycobrowser's name. It also left out a lineage with none, and added up a lineage's samples, so a lineage with more samples looked as if it had more pseudogenes. REPORT_INPUTS now hands the report every master table row with the sample it is from, and the chart shows how many pseudogenes AMAP called in each sample, by lineage, a sample with none at 0, next to a table of them. The master table explorer shows the `note` that says why a gene is a pseudogene - **A sample with no gene in a category counts 0 in the report**: `count_annot.py` writes a row for each COG or Mycobrowser category a sample has genes in, and the report plotted the rows as they came, so a sample with no gene in a category was left out of that category's figures, of the Dunn tests between lineages run on them, of the overview and of the statistics table, instead of counting 0. Its lineage's box, mean and median were those of the samples that have some, a lineage none of whose samples has any dropped out of the comparison, and the comparison with the median never showed a sample that had lost a category. Each sample now counts 0 in every category another sample of the run has genes in, and the statistics table's Count is every sample of the lineage. A category no sample has keeps its empty figure, and `*-AMAP.COG.tsv` and `*-AMAP.MycoBrowser.tsv` are unchanged -- **The clean proteome leaves out the pseudogenes AMAP called**: `export_clean_proteome.py` took them from the first of `Is_Pseudogene`, `pseudogene` and `is_pseudogene` that the master table has, which is Mycobrowser's, and looked each protein up by its whole FASTA ID, `L4_CDC1551_Rv0001;+;0-1524` where the master table has `L4_CDC1551_Rv0001`, so it never found one and its log said 0 pseudogenes. Only the proteins with an internal stop codon were left out, and the pseudogenes without one stayed in `*-AMAP.clean.faa`, which the docs say has none: 42 of the 123 AMAP calls in CDC1551 (a frameshift or lost stop codon, no stop codon within reach, no stop codon as in the reference) and 27 of the 129 in E. coli Sakai against K-12. It now leaves out the genes whose `pseudogene` is yes, found by the ID before the `;`, and on both genomes holds the CDS the GFF3 does not mark `pseudo=true`, 3891 and 5194, with the proteins the GenBank translates for them. A master table without that column stops it, where a warning let every pseudogene through +- **The clean proteome leaves out the pseudogenes AMAP called**: `export_clean_proteome.py` took them from the first of `Is_Pseudogene`, `pseudogene` and `is_pseudogene` that the master table has, which is Mycobrowser's, and looked each protein up by its whole FASTA ID, `L4_CDC1551_Rv0001;+;0-1524` where the master table has `L4_CDC1551_Rv0001`, so it never found one and its log said 0 pseudogenes. Only the proteins with an internal stop codon were left out, and the pseudogenes without one stayed in `*-AMAP.clean.faa`, which the docs say has none: 42 of the 123 AMAP calls in CDC1551 (a frameshift or lost stop codon, no stop codon within reach, no stop codon as in the reference) and 27 of the 129 in *E. coli* Sakai against K-12. It now leaves out the genes whose `pseudogene` is yes, found by the ID before the `;`, and on both genomes holds the CDS the GFF3 does not mark `pseudo=true`, 3891 and 5194, with the proteins the GenBank translates for them. A master table without that column stops it, where a warning let every pseudogene through ### ๐Ÿ› Pipeline - **Pseudogenes in the GFF3 and GenBank**: the GFF3 never said which genes were pseudogenes, and the GenBank only for genes with an internal stop, though the master table knew. Both now carry AMAP's flag with its reason (`pseudo=true` and `Note`, `/pseudo` and `/note`), and mark the ends a contig cut off (`partial=true` with `start_range` or `end_range`, `<` or `>`); the master table and `*-AMAP.tsv` have them as their last columns @@ -67,33 +67,34 @@ directly comparable with 1.0.1. - **The pangenome without lineage typing**: PANGENE aligned each sample's proteins onto the genomes PATHOTYPR renamed, so with `--skip_typing`, which every custom reference from another genus needs, it had none and wrote empty files for every sample. It now aligns them onto the sample sheet's genomes, each copied to `.fasta` first, so that two files of the same name (`assembly.fasta` from two assemblies) stay apart and a `.fna` is not passed over - **Genes not in the reference, sample by sample**: the report counted only the samples that had some, so a sample whose genes all matched the reference dropped out of its lineage's box instead of counting 0, and when no sample had any (H37Rv against itself, as in `-profile test`) the section was an empty plot. Every sample is now counted, the section says so when none has any, and each point is drawn at its count, on an axis from 0, where it was jittered up and down by as much as 0.4 - **The report's pangenome section lost a genome**: pangene names each genome `#0`, and the report read the merged matrix with R's default comment character, so its header stopped at the first `#`. One genome column was dropped and the gene names became row names, so the core/accessory and frequency plots used a single column and the heatmap never appeared. The missing-gene and duplication tables, which carry the reference annotation's gene IDs as given, are now read without a comment character too -- **Genes across the origin of a circular sequence**: Bakta reads a sequence as a complete replicon when its FASTA description says `complete` (NCBI's "complete genome" and "complete sequence") or `circular=true` (Unicycler), joins its ends and calls a gene across its origin from its start to its stop, start after end. TYPE_CASES stopped on such a gene: E. coli O157:H7 Sakai with NCBI's headers, whose stcE runs across the origin of plasmid pO157 (92563..2502), failed three times and was left out, and the run finished without failing. Past it, `check_cds.py` dropped such a gene as invalid coordinates, deduplication and the QC table gave it a negative length, the GFF3 wrote it backwards, the GenBank writer stopped on it and the Panaroo GFF3 spanned it over the whole sequence. It now goes through whole and is written as Bakta writes it: start after end in the tables, `join()` across the origin in a circular GenBank record, and in the GFF3 its end plus the sequence's length on a region marked `Is_circular=true`. A gene near the origin of such a sequence is read across it too, so Rop on plasmid pOSAK1, of which Bakta called the first 141 bp, comes out whole. Annotated against K-12, Sakai with NCBI's headers now has 5331 genes in common with the same genome under bare names, which Bakta reads as linear, and differs in stcE and Rop, whole where the linear reading has 804 of stcE's 886 residues and no Rop, and in two phage genes Bakta starts elsewhere on a complete genome -- **Typed closed genomes were annotated as linear**: PATHOTYPR renamed a genome of one sequence to `_` and nothing else, so the description Bakta reads a complete, circular replicon from never reached it, whatever the header said, and a gene across the origin of a closed genome came out as two pieces under one name, marked 5' and 3' partial. Only the sequence's ID is replaced now, and the description after it stays, as with `--skip_typing`. Typed from NCBI's headers, CDC1551, CP110619, AL123456.3 and M. bovis get the same genes as before, in place, name, flags and protein, though Bakta starts a few of its own calls elsewhere on a complete genome (dnaA at 1 instead of 34), so the gene IDs that are Bakta locus tags can be numbered differently. H37Rv turned so that the origin cuts gyrB, gyrA or pknB gives back all 4031 GenBank CDS, where it gave 4030 with the cut gene in two pieces. The GenBank record of such a genome is marked circular, with the description in its definition, and its GFF3 has a region marked `Is_circular=true` +- **Genes across the origin of a circular sequence**: Bakta reads a sequence as a complete replicon when its FASTA description says `complete` (NCBI's "complete genome" and "complete sequence") or `circular=true` (Unicycler), joins its ends and calls a gene across its origin from its start to its stop, start after end. TYPE_CASES stopped on such a gene: *E. coli* O157:H7 Sakai with NCBI's headers, whose stcE runs across the origin of plasmid pO157 (92563..2502), failed three times and was left out, and the run finished without failing. Past it, `check_cds.py` dropped such a gene as invalid coordinates, deduplication and the QC table gave it a negative length, the GFF3 wrote it backwards, the GenBank writer stopped on it and the Panaroo GFF3 spanned it over the whole sequence. It now goes through whole and is written as Bakta writes it: start after end in the tables, `join()` across the origin in a circular GenBank record, and in the GFF3 its end plus the sequence's length on a region marked `Is_circular=true`. A gene near the origin of such a sequence is read across it too, so Rop on plasmid pOSAK1, of which Bakta called the first 141 bp, comes out whole. Annotated against K-12, Sakai with NCBI's headers now has 5331 genes in common with the same genome under bare names, which Bakta reads as linear, and differs in stcE and Rop, whole where the linear reading has 804 of stcE's 886 residues and no Rop, and in two phage genes Bakta starts elsewhere on a complete genome +- **Typed closed genomes were annotated as linear**: PATHOTYPR renamed a genome of one sequence to `_` and nothing else, so the description Bakta reads a complete, circular replicon from never reached it, whatever the header said, and a gene across the origin of a closed genome came out as two pieces under one name, marked 5' and 3' partial. Only the sequence's ID is replaced now, and the description after it stays, as with `--skip_typing`. Typed from NCBI's headers, CDC1551, CP110619, AL123456.3 and *M. bovis* get the same genes as before, in place, name, flags and protein, though Bakta starts a few of its own calls elsewhere on a complete genome (dnaA at 1 instead of 34), so the gene IDs that are Bakta locus tags can be numbered differently. H37Rv turned so that the origin cuts gyrB, gyrA or pknB gives back all 4031 GenBank CDS, where it gave 4030 with the cut gene in two pieces. The GenBank record of such a genome is marked circular, with the description in its definition, and its GFF3 has a region marked `Is_circular=true` - **The report's duplication summary**: it printed the number of homologous community pairs as markdown from a chunk that did not pass its output on as markdown, so the report showed `## **48** homologous community pairs detected across all genomes.` in a code box. It is now a sentence. The example report's duplication table had columns `GET_ALLDUP` never writes, so its Network Metrics tab said there were none; the demo data now has the columns `GET_ALLDUP` writes and the example is rendered again -- **Modularity in `homologous_communities.tsv`**: `Modularity1` and `Modularity2` were computed with the whole community as its only group, which makes them 0 whatever the community, as on all 29 rows of CDC1551 against M. bovis. The two columns are gone +- **Modularity in `homologous_communities.tsv`**: `Modularity1` and `Modularity2` were computed with the whole community as its only group, which makes them 0 whatever the community, as on all 29 rows of CDC1551 against *M. bovis*. The two columns are gone - **COG categories in `homologous_communities.tsv`**: with numpy 2, as in the container, `Elements1` and `Elements2` read `{'L': np.int64(4)}`, and they counted each protein eggNOG gave no category in category `+` or `-`, since its ID keeps the strand FIX_CDS wrote. They are now plain counts, the largest first and ties in alphabetical order rather than in an order that changed from run to run, and a protein without a category counts as `-`, eggNOG's mark for none - **The order of `-profile`**: from Nextflow 25.10, whose strict parser applies the profiles in the order `-profile` names them and lets the last one win, the laptop caps of the container profiles took over from the others: `-profile slurm,singularity` held every task of a cluster run to the launching machine's CPUs and 16 GB, where `-profile singularity,slurm` gave it 48 CPUs and 128 GB, `-profile test,docker` ran the test with 16 GB and `-profile highmem,slurm` with 128 GB. No profile sets a cap now: they are set after the profiles, from whether `highmem`, `test` or a scheduler profile is on, so every order gives what Nextflow 24.04 gave. The parameter docs gave 24 CPUs and 50 GB as the defaults, which no run got -- **The duplication files are the same from one run to the next**: `homologous_communities.tsv` listed its pairs of communities in the order of a set, which changes with Python's hash seed from one run to the next, and with it which community of a pair came first and the last digit of `AvgClustering`, an average over the community's proteins that networkx went through in the order of their set: 0.5833333333333333 or 0.5833333333333334 for the same pair of CDC1551 and M. bovis. The pairs now come in the order of the communities, the sample first by name on the left, and the metrics go through the proteins in alphabetical order. Each family in `duplications_*_groups.tsv` lists its proteins in alphabetical order, where they too came in the order of a set, and each sample's two network drawings no longer start from a random layout and a random partition, so every file both steps write is the same run after run +- **The duplication files are the same from one run to the next**: `homologous_communities.tsv` listed its pairs of communities in the order of a set, which changes with Python's hash seed from one run to the next, and with it which community of a pair came first and the last digit of `AvgClustering`, an average over the community's proteins that networkx went through in the order of their set: 0.5833333333333333 or 0.5833333333333334 for the same pair of CDC1551 and *M. bovis*. The pairs now come in the order of the communities, the sample first by name on the left, and the metrics go through the proteins in alphabetical order. Each family in `duplications_*_groups.tsv` lists its proteins in alphabetical order, where they too came in the order of a set, and each sample's two network drawings no longer start from a random layout and a random partition, so every file both steps write is the same run after run - **No web browser**: both duplication steps wrote their networks with pyvis' `show()`, which also opens the file in a web browser, so a run without a container on a desktop opened a browser tab for every network. They are written as before, without being opened -- **A custom reference with a gene across the origin, or with several sequences**: `PREPARE_REFERENCE` named each reference protein after where its pieces lie, so tagA on plasmid pO157 of E. coli O157:H7 Sakai, `join(92527..92721,1..2502)` in NCBI's GenBank record, became `pO157p01_1-92721`, the whole plasmid, and lost its stop; from NCBI's GFF3, which writes it 92527..95223, it came out as its first 65 of 898 residues. Bakta's stcE on the reference, 92563..2502, matched no gene and was listed among the false positives, and the calls were compared with the genes of every sequence at once: seven calls on pO157 and two on the chromosome counted as matched by genes of the other at the same numbers, and the GenBank's 1-92721 hid two more. With Sakai as its own reference, the sample with NCBI's headers came out with tagA as 1..2502, 65 codons short and not marked partial (GenBank), or from Bakta's later start (GFF3), and with bare headers the GFF3 lost the 2502 bp piece altogether. A gene across the origin now ends past the sequence's last base, as GFF3 writes it (`pO157p01_92527-95223`, with its stop), each protein names its sequence (`contig=`), Bakta keeps the FASTA's names on the reference, and each call is compared with the genes of its own sequence, on both sides of the origin; calls on a sequence the annotation has no gene on are not judged. Both formats now give the same 5155 proteins and the same 260 false positives, and the sample with NCBI's headers gets tagA exactly, 92527..2502. K-12's 142 false positives are unchanged +- **A custom reference with a gene across the origin, or with several sequences**: `PREPARE_REFERENCE` named each reference protein after where its pieces lie, so tagA on plasmid pO157 of *E. coli* O157:H7 Sakai, `join(92527..92721,1..2502)` in NCBI's GenBank record, became `pO157p01_1-92721`, the whole plasmid, and lost its stop; from NCBI's GFF3, which writes it 92527..95223, it came out as its first 65 of 898 residues. Bakta's stcE on the reference, 92563..2502, matched no gene and was listed among the false positives, and the calls were compared with the genes of every sequence at once: seven calls on pO157 and two on the chromosome counted as matched by genes of the other at the same numbers, and the GenBank's 1-92721 hid two more. With Sakai as its own reference, the sample with NCBI's headers came out with tagA as 1..2502, 65 codons short and not marked partial (GenBank), or from Bakta's later start (GFF3), and with bare headers the GFF3 lost the 2502 bp piece altogether. A gene across the origin now ends past the sequence's last base, as GFF3 writes it (`pO157p01_92527-95223`, with its stop), each protein names its sequence (`contig=`), Bakta keeps the FASTA's names on the reference, and each call is compared with the genes of its own sequence, on both sides of the origin; calls on a sequence the annotation has no gene on are not judged. Both formats now give the same 5155 proteins and the same 260 false positives, and the sample with NCBI's headers gets tagA exactly, 92527..2502. K-12's 142 false positives are unchanged - **`pipeline_info/params.yml` is the run's own**: with -resume it came back from the cache, so it held the first run's parameters whatever later runs were given, since its script reads `params` as a whole, which Nextflow 26.10's default parser leaves out of the task's hash. And all but the first parameter fell outside `params:`, with a stray `YAML` line at the end, because Nextflow strips a script's indentation once the script is filled in. DUMP_PARAMS now never comes from the cache, and writes each value as a JSON string under `params:` - **Bakta's SLURM QoS only on SLURM**: Bakta's jobs asked for `--qos=medium` under every scheduler, though only SLURM's sbatch takes it, and under `-profile sge` it took the place of the memory request the sge profile gives every task. The slurm profile now sets it for Bakta - **The Mycobrowser table is checked whenever it is read**: validation checked `--mycobrowser_ref` only with the functional annotation, though the interactive report reads it in every run, so under `--skip_functional_annotation` a wrong path went unwarned and the report came out without Mycobrowser's categories - **The duplication networks draw offline**: both duplication steps loaded their networks' drawing library from a CDN, so without the internet the pages drew nothing. The library now goes inside each page, about 0.7 MB - **`environment.yml` pins eggNOG-mapper 2.1.13**, the version the container ships, where it said 2.1.12 -- **The pangenome shared no gene between samples**: PANGENE aligned each sample's own proteins to the other samples' genomes, ran pangene once per sample and named each column `_#0`, and REPORT_INPUTS merged those matrices on gene names that carry the sample's name and coordinates (`A4_Mbovis_Rv0001;+;0-1524`, `L4_CDC1551_Rv0001;+;0-1524`), so no row was ever in two samples and no gene was ever core. With N samples the matrix had N(N-1) columns: on the first 2 Mb of CDC1551, M. bovis and H37Rv the report showed 5379 gene families in 6 samples, all shell, with 200 genes of one genome as its heatmap, and as no genome was searched for its own genes, the interactive report found no gene in one genome alone. PANGENE now uses pangene as it is meant to be used: one set of proteins, the reference proteome and each sample's genes that no reference gene matches, aligned to every genome, the sample's own included, and one graph for the run, N alignments where there were N(N-1). A row is a reference gene (`Rv1506c`) or, outside the reference, one sample's copy of a gene (`L4_CDC1551_01396`), and a column a sample, so the same run shows 1810 genes, 1760 of them in all three, RD4 and RD10 missing from M. bovis only, the phiRv1 prophage from CDC1551 only, and three genes of the RvD2 region that CDC1551 and M. bovis carry and H37Rv lacks. Its files are now `PANGENE/pangenome.{gfa,bubbles.txt,Rtab,stat}`, one set for the run instead of one per sample, the graph statistics, which were never published, among them +- **The pangenome shared no gene between samples**: PANGENE aligned each sample's own proteins to the other samples' genomes, ran pangene once per sample and named each column `_#0`, and REPORT_INPUTS merged those matrices on gene names that carry the sample's name and coordinates (`A4_Mbovis_Rv0001;+;0-1524`, `L4_CDC1551_Rv0001;+;0-1524`), so no row was ever in two samples and no gene was ever core. With N samples the matrix had N(N-1) columns: on the first 2 Mb of CDC1551, *M. bovis* and H37Rv the report showed 5379 gene families in 6 samples, all shell, with 200 genes of one genome as its heatmap, and as no genome was searched for its own genes, the interactive report found no gene in one genome alone. PANGENE now uses pangene as it is meant to be used: one set of proteins, the reference proteome and each sample's genes that no reference gene matches, aligned to every genome, the sample's own included, and one graph for the run, N alignments where there were N(N-1). A row is a reference gene (`Rv1506c`) or, outside the reference, one sample's copy of a gene (`L4_CDC1551_01396`), and a column a sample, so the same run shows 1810 genes, 1760 of them in all three, RD4 and RD10 missing from *M. bovis* only, the phiRv1 prophage from CDC1551 only, and three genes of the RvD2 region that CDC1551 and *M. bovis* carry and H37Rv lacks. Its files are now `PANGENE/pangenome.{gfa,bubbles.txt,Rtab,stat}`, one set for the run instead of one per sample, the graph statistics, which were never published, among them ### ๐Ÿ“Š Report - **A report organised by question, `results/amap_report.html`**: next to `final_report.html`, a new process, `AMAP_REPORT`, writes one self-contained file from each genome's own outputs: a table of the genomes with the values far from comparable genomes flagged, a map of every reference gene in every genome where regions of difference show as blocks, a lookup of one gene in every genome, the correction from Bakta's calls to AMAP's genes with the genes behind each step, the pseudogenes by lineage, what lies outside the reference, and the functional profile, the categories that differ most between lineages first. It needs only what the correction leaves, so a run with `--skip_functional_annotation`, which gets no `final_report.html`, gets this one, and it names the genomes of the sample sheet that produced no annotation. Fifteen MTBC genomes make 0.5 MB (`final_report.html` is 4.4 MB for twelve); it draws everything with its own JavaScript and needs only Python's standard library to write +- **`final_report.html` opens with no network**: every report carried a script that fetched MathJax from mathjax.rstudio.com, for maths it never shows; `mathjax: null` leaves it out, so the report opens offline, as the example on the website does. It also declares its language, gives its figures alt text and its tab an icon ### ๐Ÿงน Code - **`reconcile_coordinates` is gone**: it read reference coordinates from protein identifiers that have not carried them since March 2026, so it changed nothing. The paralog check does its job with the sample's own offsets ### ๐Ÿ“š Documentation -- **A website in plain HTML, drawn with real genes**: MkDocs is gone. The front page starts a newcomer in three steps and builds the command for their machine, the pipeline is a metro map whose stations are `main.nf`'s processes, and the four cases, the missing genes and the harder cases (paralogs, small proteins, contig ends, frameshifts, copies) are drawn on genes of CDC1551, M. bovis and E. coli from the benchmark. The tutorial and user guide stay markdown, readable on GitHub, and `tools/build_docs.py` turns them into pages of the same site, dark by default +- **A website in plain HTML, drawn with real genes**: MkDocs is gone. The front page starts a newcomer in four steps and builds the command for their machine, the pipeline is a metro map whose stations are `main.nf`'s processes, and the four cases, the missing genes and the harder cases (paralogs, small proteins, contig ends, frameshifts, copies) are drawn on genes of CDC1551, *M. bovis* and *E. coli* from the benchmark. The tutorial and user guide stay markdown, readable on GitHub, and `tools/build_docs.py` turns them into pages of the same site, dark by default - **The site is checked against the code**: `tools/check_site.py` runs every drawn window through `bin/`, compares the thresholds on the pages with the constants in the code, the metro map with `main.nf` and every option with the pipeline's parameters, so a change that makes the site wrong fails the docs workflow. `tools/figure_data.py` redraws the windows from a benchmark run - **What the docs said that the pipeline does not do**: with `--skip_functional_annotation` a run still writes `AMAP/`, with the corrected table and sequences, where the outputs page said it did not; no step writes `PATHOTYPR/renamed_fastas.tsv`; and the 19-residue toxin the algorithm page names is IbsC (b4665), not an ldr toxin -- **A drawing on every page that explains something**: the duplication family and communities of CDC1551 and M. bovis, from a real run of both duplication steps; the sample sheet with its tabs and the messages the validator prints; the ranges a genome is validated against; the container and what it holds; a run on a cluster; what happens to a task that fails; the report section by section; the five outcomes and the three inputs of a decision; the command builder on the quickstart. Most are built from the code or the config, so they change when it does +- **A drawing on every page that explains something**: the duplication family and communities of CDC1551 and *M. bovis*, from a real run of both duplication steps; the sample sheet with its tabs and the messages the validator prints; the ranges a genome is validated against; the container and what it holds; a run on a cluster; what happens to a task that fails; the report section by section; the five outcomes and the three inputs of a decision; the command builder on the quickstart. Most are built from the code or the config, so they change when it does - **The benchmark on the front page**: the seven genomes whose answer is known, and how many of their genes AMAP finds, checked against `tests/benchmark/README.md`. A shared link to any page shows a card drawn from the front page's genes - **More that the docs got wrong**: the duplication page listed ten network metrics, the table has four, and its columns ending in 1 and 2 are the two communities of a pair, not two levels; Nextflow needs Java 17, not 11; `--skip_functional_annotation` leaves no GFF3 or GenBank, which come after eggNOG; tasks are capped at 16 GB on one machine and at 128 GB under a scheduler profile, not 50 GB; and lineage typing is not skipped on its own for other species, only Mycobrowser is - **The tutorial's numbers are today's**: its case counts and QC line, and their copies on the front page, the outputs page and the custom-reference page, came from runs in July. They now come from the tutorial's own commands run again on 18 October 2026 with `--skip_functional_annotation`, the QC line from GET_OUTPUTS' own script run on that run's corrected genes: CDC1551 has 4014 genes and 123 pseudogenes as AMAP calls them, where the old line counted Mycobrowser's 13, and Sakai keeps 1387 genes K-12 lacks, not 1356. The tutorial also says where `--validate_only` writes its verdict and where the number of reference proteins extracted is, as neither reaches the console @@ -102,18 +103,20 @@ directly comparable with 1.0.1. - **One DOI**: the README, CITATION.cff and every page of the website cited the DOI of one version on Zenodo, and the manifest, `main.nf` and the interactive report the concept DOI, which always resolves to the latest version. All of them cite the concept DOI now - **Bakta's options on a SLURM cluster**: the HPC page says how to give Bakta a site's options, since for Bakta the QoS the slurm profile sets takes the place of `-process.clusterOptions` - **The website is served as it is built**: GitHub Pages serves `docs/` from main, and by default runs Jekyll over it first, which would have made a page of each markdown file with front matter at the address of the page `tools/build_docs.py` builds from it. `docs/.nojekyll` turns Jekyll off, and `tools/check_site.py` fails without it +- **Website: find anything, read it at any width, and trust what it says**: a search on every page, from the header, `/` or Ctrl+K, finds any section, option, story, front-page question or changelog entry, and each option has a link of its own. Docs pages read in a wider column with their contents on every screen, code that never breaks inside a word, and a missing page that points to the nearest one. Every drawing works by keyboard, touch and screen reader and holds still while stepped through. The front page shows Bakta alone against AMAP, what a working test prints and how long it takes, and commands true for each engine. The pages say what the pipeline does today, in the same words everywhere, and `tools/check_site.py` checks those claims +- **Website: the questions a first run raises**: search answers questions asked in plain words, Troubleshooting says why a sample can be missing from a run that completed, a failed input check is described as it behaves (its report in the terminal, no `validation.log`), the laptop advice is the same on every page, and links to headings renamed since, the changelog's GitHub anchors among them, still land ### ๐Ÿงช Testing - **`tests/selftest/h37rv_selftest.sh`** annotates H37Rv with itself inside the container and fails unless every GenBank CDS comes back exactly - **`tests/test_cache_keys.py`** fails when a process runs AMAP code that is not in its cache key, or when its script names a `bin/` entry as a bare word without declaring it -- **`tests/benchmark/benchmark.sh`** scores the correction algorithm on seven genomes whose right answer is known (H37Rv twice, CDC1551 complete and in 40 contigs, M. bovis, E. coli K-12 and Sakai), downloading them and running Bakta and miniprot in the container the first time, and lists every gene that changed since an earlier run. A gene whose sequence changed counts as found when a gene with its name covers half of its own bases there, not half of the stretch it lands on, which an insertion can make far longer (63 kb with a prophage inside for b1004 in Sakai); a gene mostly deleted is not counted. Changed genes found: 50 of 50 in CP110619, 860 of 862 in CDC1551, 1515 of 1515 in M. bovis and 3363 of 3367 in Sakai, and the README says why each of the other six is not +- **`tests/benchmark/benchmark.sh`** scores the correction algorithm on seven genomes whose right answer is known (H37Rv twice, CDC1551 complete and in 40 contigs, *M. bovis*, *E. coli* K-12 and Sakai), downloading them and running Bakta and miniprot in the container the first time, and lists every gene that changed since an earlier run. A gene whose sequence changed counts as found when a gene with its name covers half of its own bases there, not half of the stretch it lands on, which an insertion can make far longer (63 kb with a prophage inside for b1004 in Sakai); a gene mostly deleted is not counted. Changed genes found: 50 of 50 in CP110619, 860 of 862 in CDC1551, 1515 of 1515 in *M. bovis* and 3363 of 3367 in Sakai, and the README says why each of the other six is not - **`tests/test_name_by_lineage.py`** runs PATHOTYPR's own script, rendered as Nextflow renders it, with a stand-in for pathotypr that writes the summaries pathotypr wrote for CDC1551 complete and in 40 contigs, and fails unless Bakta would read a renamed genome as circular exactly when it reads the original so - **`tests/test_sample_ids.py`** runs TYPE_CASES, GET_OUTPUTS, GET_COUNT, GET_DUPLICATIONS, REPORT_INPUTS and GET_ALLDUP as Nextflow renders them, chained as the pipeline chains them, for a typed and an untyped run whose sample IDs hold underscores, and then the report's own R code where R is installed, and FINAL_REPORT's own script on a run's tables and on one that leaves every table it can empty - **`tests/test_validate_inputs.py`** runs `VALIDATE_INPUTS`'s own script, rendered as Nextflow renders it, in a task directory laid out as Nextflow lays one out: genomes in a folder the task cannot see, relative paths, two genomes with the same file name and a sheet of 500 samples; and checks both databases, eggNOG's in each state `EGGNOG_DB` can leave it in - **`tests/test_eggnog_db.py`** runs `EGGNOG_DB`'s own script, rendered as Nextflow renders it, with its folder staged as Nextflow stages an input and stand-ins for eggNOG-mapper's download and build: the database is built in that folder whatever it is called, one already there is used as it is, and a short `bacteria.dmnd` is built again - **`tests/test_eggnog_annotation.py`** runs `ANNOT_EGGNOGMAP_BATCH`'s own script the same way, with a stand-in for `emapper.py` that looks for the database as eggNOG-mapper does: the database is found whatever its folder is called, a folder called `split`, where the script writes each sample's annotations, is left as it was, and the tag and log count a single sample as one - **`tests/test_reference_files.py`** runs the scripts of `BAKTA_ANNOTATOR`, `PATHOTYPR`, `GET_OUTPUTS` and `BUNDLED_REFERENCE`, rendered as Nextflow renders them, their own Groovy included, with stand-ins for Bakta and pathotypr and each reference file on the host, under a relative path or inside the container; and fails when a process reads a reference file's param other than as what it falls back on when nothing is staged; and checks the Mycobrowser table whenever a step reads it -- **`tests/test_pangene_genomes.py`** runs PANGENE_GENOME and PANGENE's scripts as Nextflow renders them on real genomes of CDC1551, M. bovis and H37Rv cut around RD4 and RvD2 (`tests/data/pangenome/`, cut again by its `make_fixture.py`), with stand-ins for miniprot, pangene, k8 and gfatools that hand names on as the real ones do, and fails unless every genome, the sample's own included, is searched with the same proteins, each column is named after its genome, typed or not, a genome none of them is found in keeps a column of zeros, and a protein two samples share goes in once. Where the real tools are installed, as in the container, RD4 must come out missing from M. bovis only and the three RvD2 genes in CDC1551 and M. bovis only, one row each +- **`tests/test_pangene_genomes.py`** runs PANGENE_GENOME and PANGENE's scripts as Nextflow renders them on real genomes of CDC1551, *M. bovis* and H37Rv cut around RD4 and RvD2 (`tests/data/pangenome/`, cut again by its `make_fixture.py`), with stand-ins for miniprot, pangene, k8 and gfatools that hand names on as the real ones do, and fails unless every genome, the sample's own included, is searched with the same proteins, each column is named after its genome, typed or not, a genome none of them is found in keeps a column of zeros, and a protein two samples share goes in once. Where the real tools are installed, as in the container, RD4 must come out missing from *M. bovis* only and the three RvD2 genes in CDC1551 and *M. bovis* only, one row each - **`tests/test_report_tables.py`** merges the matrix PANGENE writes as REPORT_INPUTS does and checks that the report reads every table a `#` can reach without a comment character; where R is installed, as in the container, it also runs the report's pangenome code on it and counts the genes all its samples share - **`tests/test_cog_counts.py`** counts COG categories as `count_annot.py` does, for genes in none, in two and in lowercase, then runs the steps after the correction as `tests/test_sample_ids.py` chains them, with eggNOG giving some genes no category, then no gene one, then some samples no gene in a category another sample has genes in, and reads what the report's own overview, per-COG figures and statistics table show where R is installed: each category in a figure of its own, with every sample, at 0 where it has no gene. The Mycobrowser figures, statistics and ratios to the median are read the same way for a sample with no gene in one of their categories - **`tests/test_duplication_report.py`** runs GET_DUPLICATIONS and GET_ALLDUP as Nextflow renders them, chained as the pipeline chains them, with eggNOG giving one copy of a duplicated gene a category, another none and the last no row at all, and checks the columns and COG counts of `homologous_communities.tsv`; where R is installed, it knits the report's duplication summary on that table and on the one `GET_ALLDUP` writes when nothing is duplicated. It also fails when a chunk of the report prints markdown without passing it on as markdown @@ -121,7 +124,7 @@ directly comparable with 1.0.1. - **`tests/test_qc_pseudogenes.py`** runs the steps after the correction as `tests/test_sample_ids.py` chains them, with FIX_CDS calling some genes pseudogenes and Mycobrowser marking another, with Mycobrowser on and off, and checks that `*-AMAP.qc.tsv`, REPORT_INPUTS and the report's own QC table count the pseudogenes AMAP called, as many as the GFF3 and the GenBank mark - **`tests/test_duplication_reproducible.py`** runs `get_duplications.py` and `network_genomes.py` under three hash seeds, on the Diamond tables of three genomes with a family whose average clustering changes with the order of its proteins, and fails unless every file they write is the same under all three; it also fails when either script opens a web browser, with hits or without, as a stand-in browser that records what it is asked to open tells, and when a network page asks a CDN for its drawing library - **`tests/test_custom_reference.py`** extracts a reference whose plasmid has a gene across its origin from a GenBank record and from GFF3 written either way, on either strand, compares Bakta's calls with it sequence by sequence, and runs `PREPARE_REFERENCE`'s own script, rendered as Nextflow renders it, with a stand-in for Bakta that names sequences as Bakta does -- **`tests/test_amap_report.py`** writes the report from four genomes of a real run cut to two stretches of the reference (`tests/data/report/`, rebuilt by its `make_fixture.py`), where M. bovis lacks RD4 and a contig end cuts dnaE2, plus a gene across the origin of a circular plasmid, and checks that every gene, and every Bakta call AMAP did not keep, comes back out of the page as it went in, with Python and, where node is installed, with the page's own JavaScript, along with the states, the flags, the correction's steps and what the process hands the script +- **`tests/test_amap_report.py`** writes the report from four genomes of a real run cut to two stretches of the reference (`tests/data/report/`, rebuilt by its `make_fixture.py`), where *M. bovis* lacks RD4 and a contig end cuts dnaE2, plus a gene across the origin of a circular plasmid, and checks that every gene, and every Bakta call AMAP did not keep, comes back out of the page as it went in, with Python and, where node is installed, with the page's own JavaScript, along with the states, the flags, the correction's steps and what the process hands the script - **`tests/test_dump_params.py`** runs DUMP_PARAMS with Nextflow where it is installed, under both parsers, with values holding quotes, `$`, backticks, a backslash and a line break, and again with -resume and one value changed; everywhere, it fails if the step can come from the cache - **`tests/test_cluster_options.py`** reads which profile gives Bakta the QoS, and with Nextflow resolves Bakta's and every task's cluster options for each scheduler, in both orders with a container and under both parsers - **`tests/test_environment_pins.py`** fails when `environment.yml` pins a tool at a version `docker/Dockerfile` does not install @@ -129,6 +132,7 @@ directly comparable with 1.0.1. - **`tests/test_clean_proteome.py`** runs the steps after the correction as `tests/test_sample_ids.py` chains them, with FIX_CDS calling some genes pseudogenes, one with a stop codon in its protein and the others without, and Mycobrowser marking a gene AMAP finds intact, with Mycobrowser on and off, and checks that `*-AMAP.clean.faa` holds every protein FIX_CDS wrote but the pseudogenes', as it wrote them: the CDS the GFF3 does not mark `pseudo=true`, with the GenBank's translations - **`tests/test_report_pseudogenes.py`** runs the steps after the correction as `tests/test_sample_ids.py` chains them, with FIX_CDS calling a different number of pseudogenes in each sample, none in one, and Mycobrowser marking a gene that every sample has intact, with Mycobrowser on and off, and checks that REPORT_INPUTS hands over each master table row with its sample; where R is installed, it runs the report's own pseudogene section on what REPORT_INPUTS made and checks that it draws each sample at the number AMAP called, the QC table's, a sample with none at 0, under a title that does not name Mycobrowser - **CI's Python 3.9 job runs on Ubuntu 24.04**: GitHub's `ubuntu-latest` became Ubuntu 26.04, for which setup-python has no Python 3.9, so the job stopped at setup before running a test. 3.9 is the container's Python, so it now runs on 24.04; 3.11 stays on the latest +- **The benchmark scores Bakta alone**: `tests/benchmark/benchmark.sh` scores Bakta's own calls of each genome by the same rules as AMAP's table, in two more columns of its README. Bakta alone finds 67 to 73% of the unchanged genes at exactly their coordinates and under their name (K-12, 90%), where AMAP finds all of them - **753 tests**, up from 92 ## [1.0.1] - 2026-07-25 @@ -147,7 +151,7 @@ than by reading it. Several changed what the pipeline outputs, so results from ### ๐Ÿ› Crashes - **`--skip_mycobrowser` and organism-agnostic mode**: the placeholder frame was built with a duplicate `Locus` column, so `pd.merge` refused to run and `GET_OUTPUTS` failed on every run without Mycobrowser -- **Miniprot PAF parsing**: pandas inferred the column count from the first row, and miniprot varies its trailing tags per alignment, so any file whose first row was narrower aborted the parse +- **miniprot PAF parsing**: pandas inferred the column count from the first row, and miniprot varies its trailing tags per alignment, so any file whose first row was narrower aborted the parse - **Biopython 1.85 and newer**: `SeqFeature.strand` was removed; `to_panaroo.py` and `extract_ref_proteins.py` now read it off the location - **`-profile test`**: pointed at a sample sheet naming paths that are not in this repository. It now annotates the reference genome bundled in the container, so it needs no data and nothing downloaded @@ -202,13 +206,13 @@ than by reading it. Several changed what the pipeline outputs, so results from - **`environment.yml`**: Conda environment with pinned versions - **Louvain seed**: fixed to 42 for reproducible community detection -### โšก Performance & HPC +### โšก Performance and HPC - **Dynamic resource scaling**: memory grows with each retry attempt - **Node-local scratch**: `--scratch true` (default on SLURM/SGE/PBS/LSF) - **Diamond block size tuning**: auto-calculated from available RAM - **Bakta local temp**: `--tmp-dir` on node-local disk (avoids NFS metadata storms) - **eggNOG batch mode**: single run for all samples (10ร— faster, 50ร— less filesystem I/O) -- **Thread utilization**: Bakta 12, Diamond 6-24, Miniprot 2ร—3, Pangene 2ร—N +- **Thread utilization**: Bakta 12, Diamond 6-24, miniprot 2ร—3, pangene 2ร—N - **SLURM/SGE/PBS/LSF profiles**: with time limits, queue sizes, rate limiting - **`storeDir` for eggNOG DB**: persists across runs, survives `nextflow clean` @@ -219,7 +223,7 @@ than by reading it. Several changed what the pipeline outputs, so results from - **awk `close()`**: prevents file handle exhaustion on 500+ sample eggNOG splits - **JVM tuning docs**: NXF_OPTS guidance for large runs -### ๐Ÿงฌ Organism-Agnostic +### ๐Ÿงฌ Organism-agnostic - **Custom reference**: `--ref_annotation` + `--ref_genome` replaces built-in H37Rv - **Auto-detect mode**: `--skip_mycobrowser` auto-enabled when genus โ‰  Mycobacterium - **Dynamic reference comparison**: report uses median distribution (not hardcoded H37Rv) @@ -227,13 +231,13 @@ than by reading it. Several changed what the pipeline outputs, so results from - **Generic labels**: "Ratio to Median" instead of "Ratio to H37Rv" - **Manifest updated**: "Assisted Microbial Annotator Pipeline" -### ๐Ÿ“Š Output Quality +### ๐Ÿ“Š Output quality - **Clean proteome**: `{sample}-AMAP.clean.faa` without pseudogenes or internal stops - **Annotation QC**: per-sample metrics (genome size, GC%, coding density, hypothetical %, annotation rate) - **GFF3 spec compliance**: `##sequence-region` pragmas, attribute escaping, sorted features - **QC report section**: interactive table + coding density / hypothetical % / genome size plots -### ๐Ÿงช Code Quality +### ๐Ÿงช Code quality - **58 unit tests**: covering IDs, overlaps, reconciliation, validation, clean proteome, QC, merge - **GitHub Actions CI**: Python 3.9 + 3.11 matrix (currently disabled, ready to activate) - **Type hints**: core modules with `Coord`, `Coord3`, `CoordDict` aliases @@ -251,13 +255,12 @@ than by reading it. Several changed what the pipeline outputs, so results from - **CHANGELOG**: this file - **Output tree**: documents `.clean.faa`, `.qc.tsv`, `pipeline_info/` new files -### ๐Ÿ› Bug Fixes (85 total) -- 20 critical (๐Ÿ”ด) + 65 improvements (๐ŸŸก) -- See commit history on `fix` branch for full details +### ๐Ÿ› Bug fixes (85 total) +- 20 critical fixes and 65 improvements ## [0.1.0] - 2025 - Initial release - MTBC-focused annotation pipeline -- Bakta + Miniprot + eggNOG integration +- Bakta + miniprot + eggNOG integration - HTML report with COG, Mycobrowser, pangenome analysis diff --git a/README.md b/README.md index 6293ced..e575be4 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ __Paula Ruiz-Rodriguez1__ __and Mireia Coscolla1__
- 1. I2SysBio, University of Valencia-CSIC, FISABIO Joint Research Unit Infection and Public Health, Valencia, Spain + 1. I2SysBio, University of Valencia-CSIC, FISABIO Joint Research Unit Infection and Public Health, Paterna, Spain @@ -87,14 +87,14 @@ lines are what travels between them. On the website the map is 1. **Database setup** ([eggNOG](https://github.com/eggnogdb/)): builds the eggNOG database if it is not already there. 2. **MTBC lineage typing** ([pathotypr](https://github.com/PathoGenOmics-Lab/pathotypr)): assigns lineages from phylogenetic markers and renames the genomes accordingly. Turned off with `--skip_typing`. -3. **Genome annotation** ([Bakta](https://github.com/oschwengers/bakta)): CDS, tRNAs and other features, the baseline that AMAP corrects. +3. **Genome annotation** ([Bakta](https://github.com/oschwengers/bakta)): CDS, rRNAs and other features (tRNAs are skipped), the baseline that AMAP corrects. 4. **Protein alignment** ([miniprot](https://github.com/lh3/miniprot)): the reference proteome aligned onto each genome. 5. **CDS case classification**: every CDS sorted by how it relates to the reference (novel, perfect match, imperfect match, conflicting coordinates, or a reference gene the annotator missed). 6. **Annotation correction**: conflicts resolved, coordinates adjusted and every gene translated into a corrected gene set. 7. **Functional annotation** ([eggNOG-mapper](https://github.com/eggnogdb/eggnog-mapper)): COG, KEGG, GO, EC, PFAM and CAZy assignments, and the corrected GFF3 and GenBank. 8. **Gene duplication analysis**: duplication families per genome, plus homologous communities across genomes with network metrics. 9. **Pangenome analysis** ([pangene](https://github.com/lh3/pangene)): gene content compared across the whole set. -10. **Report**: a single self-contained HTML file covering all of the above. +10. **Reports**: `amap_report.html` in every run, and `final_report.html` after eggNOG, each one self-contained HTML file. Mycobrowser's functional categories describe tuberculosis genes, so they are skipped on their own when a custom reference is not a *Mycobacterium*; lineage @@ -106,17 +106,20 @@ You need [Nextflow](https://www.nextflow.io/) 24.04 or newer and a container engine. Nothing else: every tool lives in the AMAP image, and there is no need to clone this repository. +Nextflow needs Java 17 or newer, and Conda brings both: + ```bash -conda create -n nextflow-env +conda create -n nextflow-env -c conda-forge -c bioconda nextflow conda activate nextflow-env -conda install -c bioconda nextflow ``` -Check the installation. This annotates the reference genome bundled in the -container, so you need no data of your own: +Check the installation. The test profile annotates the H37Rv genome bundled in +the image, so you need no data of your own, and `--skip_functional_annotation` +leaves out eggNOG, so you need no database either: ```bash -nextflow run PathoGenOmics-Lab/AMAP -profile docker,test +nextflow run PathoGenOmics-Lab/AMAP -profile docker,test \ + --skip_functional_annotation --outdir amap-test ``` Write a tab-separated sample sheet: @@ -152,14 +155,16 @@ for the full set of profiles and options. ## Results -Everything lands under `--outdir`, `results/` by default. The three to open -first: +Everything lands under `--outdir`, `results/` by default. Lineage typing puts +the lineage in front of each sample's name, so `CDC1551` becomes `L4_CDC1551`, +and `--skip_typing` leaves the sample ID alone. The ones to open first: | What | Where | | :--- | :--- | -| The full interactive report | `results/final_report.html` | -| Corrected annotation, per sample | `results/AMAP/{sampleId}/*-AMAP.gff3` | -| Proteins for downstream tools | `results/AMAP/{sampleId}/*-AMAP.clean.faa` | +| Each genome and gene, question by question, in every run | `results/amap_report.html` | +| The full report, after a run with eggNOG | `results/final_report.html` | +| Corrected annotation, per sample, after eggNOG | `results/AMAP/{lineage}_{sampleId}/*-AMAP.gff3` | +| Proteins for downstream tools, after eggNOG | `results/AMAP/{lineage}_{sampleId}/*-AMAP.clean.faa` | Alongside them, `results/pipeline_info/` holds `software_versions.yml` and `params.yml`, which are what belongs in a methods section. Every directory is @@ -194,7 +199,7 @@ See author = {Ruiz-Rodriguez, Paula and Coscolla, Mireia}, doi = {10.5281/zenodo.8024446}, license = {GPL-3.0}, - month = apr, + month = jul, title = {{AMAP}}, url = {https://github.com/PathoGenOmics-Lab/AMAP}, version = {1.0.1}, diff --git a/bin/report/header.html b/bin/report/header.html index 55098df..8488a2c 100644 --- a/bin/report/header.html +++ b/bin/report/header.html @@ -11,3 +11,10 @@ PathoGenOmics Lab + + + + diff --git a/bin/report/report_AMAP.Rmd b/bin/report/report_AMAP.Rmd index f7865aa..a99a6b0 100644 --- a/bin/report/report_AMAP.Rmd +++ b/bin/report/report_AMAP.Rmd @@ -2,6 +2,8 @@ title: "AMAP: Assisted Microbial Annotation Pipeline Report" author: "`r paste0('AMAP ', params$version)`" date: "`r format(Sys.time(), '%d/%m/%Y')`" +# The page's language, for screen readers and the browser's own tools. +lang: en output: html_document: @@ -16,6 +18,9 @@ output: # Everything is inlined, so the file opens with no network and survives # being emailed around. Nothing here may reference a remote URL. self_contained: true + # No maths in the report, and MathJax cannot be inlined: left on, every + # report carried a script that fetched it from mathjax.rstudio.com. + mathjax: null includes: before_body: header.html params: @@ -51,6 +56,7 @@ params: overlay.className = "zoomDiv"; var big = document.createElement("img"); big.className = "zoomImg"; + big.alt = ""; overlay.appendChild(big); document.body.insertBefore(overlay, document.body.firstChild); @@ -64,6 +70,7 @@ params: if (t.tagName !== "IMG") return; if (t.classList.contains("zoomImg")) { close(); return; } big.src = t.getAttribute("src"); + big.alt = t.getAttribute("alt") || ""; big.style.width = "100%"; Object.assign(overlay.style, { opacity: "1", width: "auto", border: "1px solid white", @@ -167,7 +174,7 @@ interactive_plot <- function(p) { } # ------------- GLOBAL VARIABLES ------------- -# Lineage color palette โ€” includes MTBC lineages and a broad auto-extension +# Lineage color palette: includes MTBC lineages and a broad auto-extension # mechanism for any organism. Unknown lineage names get colors from a # qualitative palette, ensuring the report works out-of-the-box for non-MTBC. mtbc_colors <- c("L5"="#995200","L6"="#1eb040","A1"="#d1ae00","A4"="#ff9cdb","A2"="#8ef5c8", @@ -290,7 +297,7 @@ if (length(unknown_lineages) > 0) { colors <- c(colors, extra_cols) } -# Mycobrowser โ€” may be empty for non-MTBC organisms +# Mycobrowser: may be empty for non-MTBC organisms has_mycobrowser_data <- file.exists(params$myco_file) && file.info(params$myco_file)$size > 10 if (has_mycobrowser_data) { myco_content <- tryCatch(readLines(params$myco_file, n=2), error=function(e) "") @@ -344,7 +351,7 @@ if (nrow(datos_myco) > 0) { merged_data <- data.frame() } } else { - # No Mycobrowser data โ€” create empty placeholders + # No Mycobrowser data: create empty placeholders merged_data <- data.frame() } @@ -448,7 +455,7 @@ if (has_dups) { This section provides a general overview of the dataset, showing the distribution of sequences across lineages/groups. -```{r lineage_summary, echo=FALSE, fig.align='center', fig.width=14, message=FALSE, warning=FALSE} +```{r lineage_summary, echo=FALSE, fig.align='center', fig.width=14, message=FALSE, warning=FALSE, fig.alt='Bar chart of how many genomes each lineage has, next to a ring of each lineage\'s share of the genomes.'} lineage_summary <- data %>% rename(Lineages = V4) %>% mutate(Lineages = ifelse(is.na(Lineages) | Lineages == "", "NA", as.character(Lineages))) %>% @@ -528,7 +535,7 @@ if (has_qc) { ``` ### Coding Density -```{r qc_coding_density, echo=FALSE, fig.align='center', fig.width=10, fig.height=5, message=FALSE, warning=FALSE} +```{r qc_coding_density, echo=FALSE, fig.align='center', fig.width=10, fig.height=5, message=FALSE, warning=FALSE, fig.alt='Bar chart of each sample\'s coding density, in percent, with the mean as a dashed red line.'} if (has_qc && !is.null(qc_data) && nrow(qc_data) > 0 && "coding_density_pct" %in% colnames(qc_data)) { p <- ggplot(qc_data, aes(x=reorder(sample, coding_density_pct), y=coding_density_pct)) + geom_col(fill="#2196F3", alpha=0.7) + @@ -541,7 +548,7 @@ if (has_qc && !is.null(qc_data) && nrow(qc_data) > 0 && "coding_density_pct" %in ``` ### Hypothetical % -```{r qc_hypothetical, echo=FALSE, fig.align='center', fig.width=10, fig.height=5, message=FALSE, warning=FALSE} +```{r qc_hypothetical, echo=FALSE, fig.align='center', fig.width=10, fig.height=5, message=FALSE, warning=FALSE, fig.alt='Bar chart of each sample\'s share of hypothetical proteins, in percent, with the mean as a dashed red line.'} if (has_qc && !is.null(qc_data) && nrow(qc_data) > 0 && "hypothetical_pct" %in% colnames(qc_data)) { p <- ggplot(qc_data, aes(x=reorder(sample, -hypothetical_pct), y=hypothetical_pct)) + geom_col(fill="#FF9800", alpha=0.7) + @@ -554,7 +561,7 @@ if (has_qc && !is.null(qc_data) && nrow(qc_data) > 0 && "hypothetical_pct" %in% ``` ### Genome Size Distribution -```{r qc_genome_size, echo=FALSE, fig.align='center', fig.width=10, fig.height=5, message=FALSE, warning=FALSE} +```{r qc_genome_size, echo=FALSE, fig.align='center', fig.width=10, fig.height=5, message=FALSE, warning=FALSE, fig.alt='Histogram of the genome sizes in megabases, with the mean as a dashed red line.'} if (has_qc && !is.null(qc_data) && nrow(qc_data) > 0 && "genome_size" %in% colnames(qc_data)) { qc_data$genome_mb <- qc_data$genome_size / 1e6 p <- ggplot(qc_data, aes(x=genome_mb)) + @@ -569,7 +576,7 @@ if (has_qc && !is.null(qc_data) && nrow(qc_data) > 0 && "genome_size" %in% colna ## CDS Distribution per Lineage -```{r lineage_cds_distribution, echo=FALSE, fig.align='center', fig.width=12, message=FALSE, warning=FALSE} +```{r lineage_cds_distribution, echo=FALSE, fig.align='center', fig.width=12, message=FALSE, warning=FALSE, fig.alt='Box plot of the number of CDS per genome in each lineage, each genome a point.'} p_cds_dist <- ggplot(data %>% filter(V1 == "total_cds"), aes(x = V4, y = V2, color = V4)) + geom_boxplot(outlier.shape = NA, fill = NA) + geom_jitter(alpha = 0.4, width = 0.2) + @@ -581,7 +588,7 @@ print(p_cds_dist) ## CDS not in Reference -```{r cds_NoH37Rv, echo=FALSE, fig.align='center', fig.width=12, message=FALSE, warning=FALSE} +```{r cds_NoH37Rv, echo=FALSE, fig.align='center', fig.width=12, message=FALSE, warning=FALSE, fig.alt='Box plot of how many CDS of each genome name no reference gene, by lineage, each genome a point.'} # Genome (the sample ID) and lineage of every gene not in the reference. # REPORT_INPUTS strips each file's header, so every line is a gene. datos_noH <- read.table(params$empty_file, sep="\t", dec=".", header=F, fill=T, stringsAsFactors=F, quote="", col.names=paste0("V",1:2)) @@ -609,7 +616,7 @@ if (sum(resultado$n) == 0) { ## CDS Correction Summary -```{r cds_correction_summary, echo=FALSE, fig.align='center', fig.width=12, message=FALSE, warning=FALSE} +```{r cds_correction_summary, echo=FALSE, fig.align='center', fig.width=12, message=FALSE, warning=FALSE, fig.alt='Stacked bars, one per lineage, splitting its CDS into those that matched the reference and those AMAP corrected, in percent.'} correction_data <- data %>% filter(V1 %in% c("case4","case2","case3","rv_missing")) %>% mutate(CorrectionStatus = case_when( @@ -639,7 +646,7 @@ Reference genes absent from the Bakta annotation and recovered by AMAP. Which Rv genes are missing in which lineages? Darker = missing in more genomes of that lineage. -```{r rvmissing_heatmap, echo=FALSE, fig.align='center', fig.height=12, fig.width=14, message=FALSE, warning=FALSE} +```{r rvmissing_heatmap, echo=FALSE, fig.align='center', fig.height=12, fig.width=14, message=FALSE, warning=FALSE, fig.alt='Horizontal bars for the 50 reference genes missing from the most genomes, coloured by lineage, each bar as long as the number of genomes missing that gene.'} if (has_rvmissing) { # Count how many genomes are missing each Rv gene, broken down by lineage rv_counts <- rvmissing_raw %>% @@ -720,7 +727,7 @@ make_dt(stats_cog, "cog_statistics") ``` ### Per COG (% Percentage) {.tabset} -```{r scatter_cogs_pct, echo=FALSE, results='asis', message=FALSE, warning=FALSE} +```{r scatter_cogs_pct, echo=FALSE, results='asis', message=FALSE, warning=FALSE, fig.alt=paste0('Box plot of the percentage of CDS in COG category ', cog_names, ', per genome and by lineage, with Dunn\'s tests between lineages.')} for (cat_code in names(cog_names)) { cat(paste("####", cat_code, "\n\n")) print(create_scatter_plot(datos_cog1, cat_code, 'Percentage', 'Percentage', cog_names[cat_code])) @@ -729,7 +736,7 @@ for (cat_code in names(cog_names)) { ``` ### Per COG (Nยบ CDS) {.tabset} -```{r scatter_cogs_count, echo=FALSE, results='asis', message=FALSE, warning=FALSE} +```{r scatter_cogs_count, echo=FALSE, results='asis', message=FALSE, warning=FALSE, fig.alt=paste0('Box plot of the number of CDS in COG category ', cog_names, ', per genome and by lineage, with Dunn\'s tests between lineages.')} for (cat_code in names(cog_names)) { cat(paste("####", cat_code, "\n\n")) print(create_scatter_plot(datos_cog1, cat_code, 'Count', 'Number of CDS', cog_names[cat_code])) @@ -753,7 +760,7 @@ interactive_plot(ggplotly(p) %>% layout(autosize=F, width=800, height=450)) ``` ### Per Category (%) {.tabset} -```{r myco_split_pct, echo=FALSE, results='asis', message=FALSE, warning=FALSE} +```{r myco_split_pct, echo=FALSE, results='asis', message=FALSE, warning=FALSE, fig.alt='Box plot of the percentage of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn\'s tests between lineages.'} if (!has_mycobrowser_data) { cat("Not available.\n") } else { myco_cats <- setdiff(unique(na.omit(datos_myco$Functional_Category)), "NA") for (cat_name in myco_cats) { @@ -765,7 +772,7 @@ for (cat_name in myco_cats) { ``` ### Per Category (Nยบ CDS) {.tabset} -```{r myco_split_count, echo=FALSE, results='asis', message=FALSE, warning=FALSE} +```{r myco_split_count, echo=FALSE, results='asis', message=FALSE, warning=FALSE, fig.alt='Box plot of the number of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn\'s tests between lineages.'} if (!has_mycobrowser_data) { cat("Not available.\n") } else { for (cat_name in myco_cats) { cat(paste("####", cat_name, "\n\n")) @@ -802,7 +809,7 @@ interactive_plot(ggplotly(p) %>% layout(autosize=F, width=800, height=450)) ``` ### Per Category {.tabset} -```{r ref_comparison_split, echo=FALSE, results='asis', message=FALSE, warning=FALSE} +```{r ref_comparison_split, echo=FALSE, results='asis', message=FALSE, warning=FALSE, fig.alt='Box plot of each genome\'s share of the Mycobrowser category named in its title, as a percentage of the median of all samples, by lineage.'} if (!has_mycobrowser_data || nrow(merged_data) == 0) { cat("Not available.\n") } else { ref_cats <- setdiff(unique(na.omit(merged_data$Functional_Category)), "NA") for (cat_name in ref_cats) { @@ -829,7 +836,7 @@ make_dt(stats_ref, "reference_comparison") *PE/PPE genes are characteristic of Mycobacteria. This section is only shown when Mycobrowser functional categories are available.* ### Distribution per Lineage -```{r peppe, echo=FALSE, fig.align='center', fig.width=12, fig.height=6, message=FALSE, warning=FALSE} +```{r peppe, echo=FALSE, fig.align='center', fig.width=12, fig.height=6, message=FALSE, warning=FALSE, fig.alt='Bar chart of the share of each lineage\'s CDS that are PE/PPE genes, with how many they are.'} if (!has_mycobrowser_data) { cat("PE/PPE analysis requires functional category data (Mycobrowser).\n") } else if (has_mastertable && "Functional_Category" %in% colnames(mastertable)) { peppe_count <- mastertable %>% @@ -866,7 +873,7 @@ if (has_mastertable && exists("peppe_count")) { Distribution of KEGG pathway annotations across lineages. ### Top Pathways -```{r kegg_top, echo=FALSE, fig.align='center', fig.width=14, fig.height=8, message=FALSE, warning=FALSE} +```{r kegg_top, echo=FALSE, fig.align='center', fig.width=14, fig.height=8, message=FALSE, warning=FALSE, fig.alt='Horizontal bars for the 25 KEGG pathways with the most genes, one bar per lineage.'} if (has_mastertable && "KEGG_Pathway" %in% colnames(mastertable)) { kegg_data <- mastertable %>% filter(!is.na(KEGG_Pathway) & KEGG_Pathway != "" & KEGG_Pathway != "-") %>% @@ -913,7 +920,7 @@ if (has_mastertable && exists("kegg_data")) { The genes AMAP calls pseudogenes in each sample, by lineage: internal stop codons, a frameshift or lost stop that makes the frame run on past the end of the reference protein, no stop codon within reach, or a stop the reference itself lacks. They are the genes the GFF3 marks `pseudo=true` and the QC table counts, and the master table's `note` column says why each one is a pseudogene. A sample with none counts 0. Mycobrowser's `Is_Pseudogene` says whether a gene is a pseudogene in H37Rv, not in the sample, and is not counted. ### By Lineage -```{r pseudogenes, echo=FALSE, fig.align='center', fig.width=12, fig.height=6, message=FALSE, warning=FALSE} +```{r pseudogenes, echo=FALSE, fig.align='center', fig.width=12, fig.height=6, message=FALSE, warning=FALSE, fig.alt='Box plot of the pseudogenes AMAP called per sample, by lineage, each sample a point.'} if (!has_mastertable) { cat("Master table not available.\n") } else if (!all(c("sample", "pseudogene") %in% colnames(mastertable))) { @@ -965,7 +972,7 @@ Gene duplications detected by all-vs-all Diamond BLASTp within and across genome ## Duplication Summary -```{r dup_summary, echo=FALSE, results='asis', fig.align='center', fig.width=12, fig.height=6, message=FALSE, warning=FALSE} +```{r dup_summary, echo=FALSE, results='asis', fig.align='center', fig.width=12, fig.height=6, message=FALSE, warning=FALSE, fig.alt='Histogram of community sizes: how many communities have each number of genes.'} if (has_dups && nrow(dup_data) > 0) { cat(paste0("**", nrow(dup_data), "** homologous community pairs detected across all genomes.\n\n")) @@ -1014,7 +1021,7 @@ Gene presence/absence across samples based on pangene analysis. ## Core vs Accessory Genes -```{r pangene_core, echo=FALSE, fig.align='center', fig.width=12, fig.height=8, message=FALSE, warning=FALSE} +```{r pangene_core, echo=FALSE, fig.align='center', fig.width=12, fig.height=8, message=FALSE, warning=FALSE, fig.alt='Bar chart of the pangenome\'s gene families split into core, soft core, shell and cloud, with counts and percentages.'} if (has_pangene) { # Rtab is a presence/absence matrix: gene ร— sample gene_names <- pangene_raw[, 1] @@ -1056,7 +1063,7 @@ if (has_pangene) { ## Gene Frequency Spectrum -```{r pangene_spectrum, echo=FALSE, fig.align='center', fig.width=12, fig.height=6, message=FALSE, warning=FALSE} +```{r pangene_spectrum, echo=FALSE, fig.align='center', fig.width=12, fig.height=6, message=FALSE, warning=FALSE, fig.alt='Histogram of gene families by how many samples have them, with a dashed red line at all samples, the core.'} if (has_pangene) { freq_df <- data.frame(frequency = gene_freq) @@ -1074,7 +1081,7 @@ if (has_pangene) { ## Presence/Absence Heatmap -```{r pangene_heatmap, echo=FALSE, fig.align='center', fig.height=10, fig.width=14, message=FALSE, warning=FALSE} +```{r pangene_heatmap, echo=FALSE, fig.align='center', fig.height=10, fig.width=14, message=FALSE, warning=FALSE, fig.alt='Heatmap of which samples have each variable gene family, present in green and absent in white, rows and columns clustered.'} if (has_pangene && ncol(pa_matrix) >= 2 && nrow(pa_matrix) >= 2) { # Show only shell + cloud genes (variable genes are more interesting) variable_idx <- which(gene_freq < n_samples & gene_freq >= 1) @@ -1093,7 +1100,7 @@ if (has_pangene && ncol(pa_matrix) >= 2 && nrow(pa_matrix) >= 2) { clustering_distance_rows="binary", clustering_distance_cols="binary") }, error = function(e) cat("Could not generate heatmap.\n")) } else { - cat("Fewer than 2 variable genes โ€” no heatmap needed.\n") + cat("Fewer than 2 variable genes, so no heatmap.\n") } } else { cat("No pangene data available or too few samples.\n") @@ -1106,7 +1113,7 @@ Detailed breakdown of the CDS-correction cases identified by AMAP. **Classification:** -- **Case 1 keep:** CDS missing from the reference โ€” potentially real novel genes +- **Case 1 keep:** CDS missing from the reference, potentially real novel genes - **Case 1 remove:** CDS falsely introduced by Bakta, absent from reference - **Case 2:** Perfect coordinate match with reference - **Case 3:** Same start/end but containing mismatches @@ -1114,7 +1121,7 @@ Detailed breakdown of the CDS-correction cases identified by AMAP. - **Missing Genes:** Reference genes recovered by AMAP (not annotated by Bakta) ## Boxplot -```{r cds_boxplot, echo=FALSE, fig.align='center', fig.height=5, fig.width=14, message=FALSE, warning=FALSE} +```{r cds_boxplot, echo=FALSE, fig.align='center', fig.height=5, fig.width=14, message=FALSE, warning=FALSE, fig.alt='Box plots of the number of CDS in each case, one panel per case and a box per lineage, with Dunn\'s tests between lineages.'} counts <- data %>% group_by(V4, V1) %>% summarise(n=n(), y_pos=max(V2)+1, .groups='drop') valid_groups <- data %>% @@ -1146,7 +1153,7 @@ print(p) ``` ## Violinplot -```{r cds_violin, echo=FALSE, fig.align='center', fig.height=5, fig.width=14, message=FALSE, warning=FALSE} +```{r cds_violin, echo=FALSE, fig.align='center', fig.height=5, fig.width=14, message=FALSE, warning=FALSE, fig.alt='Violin plots of the number of CDS in each case, one panel per case and a violin per lineage, with the mean and its confidence interval.'} means <- data %>% group_by(V4, V1) %>% summarise(mean_value=mean(V2), y_pos=max(V2)+1, .groups='drop') p <- ggplot(data, aes(x=V4, y=V2, color=V4)) + @@ -1161,7 +1168,7 @@ print(p) ``` ## Histogram Cases -```{r cds_histo, echo=FALSE, fig.align='center', fig.height=4, fig.width=8, message=FALSE, warning=FALSE} +```{r cds_histo, echo=FALSE, fig.align='center', fig.height=4, fig.width=8, message=FALSE, warning=FALSE, fig.alt='Stacked bars, one per lineage, of the share of its mean CDS count in each case.'} filtered_data <- means %>% filter(str_starts(V1, "case")) colores_cases <- c("case1_kept"="#13ff03","case1_removed"="#377eb8", "case2"="#984ea3","case3"="#ffff33","case4"="#ff7f00") @@ -1174,7 +1181,7 @@ ggplot(filtered_data, aes(x=V4, y=mean_value, fill=V1)) + ``` ## Heatmap -```{r cds_heatmap, echo=FALSE, fig.align='center', fig.height=5, fig.width=7, message=FALSE, warning=FALSE} +```{r cds_heatmap, echo=FALSE, fig.align='center', fig.height=5, fig.width=7, message=FALSE, warning=FALSE, fig.alt='Heatmap of the log mean number of CDS in each case, lineage by lineage.'} ggplot(means, aes(x=V4, y=V1, fill=log(mean_value))) + geom_tile() + scale_fill_viridis_c(option="C", name="Log Mean Value") + theme_minimal() + diff --git a/bin/report/styles.css b/bin/report/styles.css index e1eabb3..d19a594 100644 --- a/bin/report/styles.css +++ b/bin/report/styles.css @@ -99,10 +99,11 @@ body #TOC .amap-toc-title{ background:var(--amap-teal-dark); border-radius:7px 7px 0 0; } +/* White on the darker teal: 5.3:1, where the lighter one gave 3.3:1. */ body #TOC .tocify-item.active, body #TOC .list-group-item.active{ - background:var(--amap-teal)!important; - border-color:var(--amap-teal)!important; + background:var(--amap-teal-dark)!important; + border-color:var(--amap-teal-dark)!important; color:#fff!important; } body #TOC .list-group-item:hover{ diff --git a/docs/404.html b/docs/404.html index 3df2b74..0d9f64f 100644 --- a/docs/404.html +++ b/docs/404.html @@ -3,12 +3,13 @@ -Not here | AMAP +Page not found | AMAP + - + @@ -16,33 +17,46 @@ - + - + - - + + +
- AMAP + AMAP
-
-
+
+

That page is not here

The link that brought you here points at something that has moved or never existed. Nothing is broken on your side.

+ +

Search for it:

+
+ + +
-
+

That page is not here

no page aligns here -
Case 1, for a page: nothing aligns where this link points.
+ +
+
Case 1, for a page: nothing aligns where this link points.
The front page @@ -61,5 +85,91 @@

That page is not here

+ + + diff --git a/docs/about/changelog.html b/docs/about/changelog.html index 2337032..53ed214 100644 --- a/docs/about/changelog.html +++ b/docs/about/changelog.html @@ -4,156 +4,169 @@ Changelog | AMAP docs - + + - + - + - + - +
- AMAP + AMAP
- + + +
-

Docs / About

+

Docs / About

Describes AMAP on main, ahead of 1.0.1: what changed

Changelog

-

All notable changes to AMAP are documented in this file.

-

[Unreleased]#

+

What changed in each version.

+
On this page
+ +

On main, not yet released

#

Annotating H37Rv against its own reference now gives back every CDS of the AL123456.3 GenBank record at exactly its coordinates, strand and name: 4031 of 4031, against 4020 before. The changes apply to every genome, so results are not directly comparable with 1.0.1.

-

๐Ÿ”ฌ Correctness#

-
  • Paralog copies no longer take another gene's place: a protein with a paralog was placed at both loci, and which copy survived came down to insertion order. On H37Rv itself that lost Rv1150, Rv1945 and Rv2828c; on M. bovis it reported genes the strain lacks (the mce3 operon of RD7, the phiRv2 prophage of RD11, plcBC of RD5) as copies sitting on their paralogs. Each placement is now checked against where its protein belongs, predicted from the sample's own offset to the reference so that indels do not throw it off
  • Strand of a resolved conflict: taken from the protein's first placement anywhere in the genome instead of the one being resolved, so esxM (Rv1792), Rv3466 and two IS6110 copies came out at their own coordinates on the wrong strand; a Bakta call chosen over an alignment also took the alignment's strand
  • Reference pseudogenes keep their extent: PE21 (Rv2099c) and PPE48 (Rv3022c), which the reference annotates without a stop, were extended to the next one, and oxyR' (Rv2427A) and Rv3216 lost the partial final codon the reference gives them
  • The 1.0.1 fix for reference pseudogenes never fired: it looked for reference coordinates in a column that has not carried them since March 2026, and its tests wrote them in by hand. check_cds.py now reads the reference proteome instead
  • Alignments that stop short are completed: miniprot can miss a protein's N-terminus (Rv3611 on CP110619) or leave a repetitive C-terminus unaligned further from its stop than the 300 bp searched; the reference protein now says how much is missing
  • Ties between copies of a gene go to the one Bakta also called, then to the one where the protein belongs by synteny, not the one that comes first in the genome (E. coli K-12 b2635)
  • Draft assemblies: genes were compared on coordinates alone, and every contig starts at 1, so cut into 40 contigs CDC1551 had 87% of its proteins translated from the wrong contig. Every gene is now matched, translated and written on its own contig, and Bakta runs with --keep-contig-headers so that its contig names match the FASTA's
  • Case 4 is grouped strand by strand: about 8% of its groups (101 of 1255 on H37Rv) put an alignment on one strand together with a Bakta call on the other
  • A complete alignment is not displaced by a longer call in its frame: such a call ends at the same stop and only starts further upstream (plcD on CDC1551). Nor does a call that spans several reference genes win against them (Rv3022c and Rv3021c, one reading frame in CDC1551)
  • Reference genes are protected again: the check looked for coordinates in identifiers that no longer carry them, so any call of similar length could displace a reference gene (Rv1614)
  • Missing genes are recovered by what covers them: genes stored under Bakta locus tags looked missing by name, so a placement a codon off its gene went back in as a second copy. With the case 4 changes, ORFs annotated twice fall from 10 to 3 on CDC1551 and from 16 to 6 on M. bovis, where the reference has none
  • Real copies of a gene are kept: deduplicate_rvs.py deleted every copy of a gene but one, 34 genes on E. coli O157:H7 Sakai, which were then missing from the annotation and from the duplication analysis. Further copies get identifiers of their own
  • Genes that lost their stop no longer swallow the next gene: check_cds.py extended a gene without a stop to the next one in its frame, which after an indel is the next gene's, so that gene was annotated a second time in the same frame (M. bovis Rv0590 over Rv0590A, Rv3373 over Rv3374, Rv2293c over Rv2292c). Such a gene now keeps its extent as a pseudogene, unless the extension only restores residues its reference protein has. ORFs annotated twice fall from 3 to 2 on CDC1551 and from 6 to 3 on M. bovis, and none of those left comes from an extension
  • Small proteins: miniprot placed none of the 33 proteins of 8 to 29 residues in E. coli K-12, and put the toxin IbsC (b4665) only on its near-identical neighbour IbsD. Every protein without an exact placement is now looked for in the six-frame translation of the genome, which takes K-12 against itself from 4252 to all 4286 of its genes, its alternative-start isoforms counted with their gene, and Sakai from 198 to 220 of 220
  • Frameshifts and lost stops: a gene was extended to the next stop in its frame however far that was, and came out as a complete gene with a C-terminus its reference lacks; 18 in CDC1551 and 20 in M. bovis. It now keeps its extent and is flagged as a pseudogene when that stop lies more than ten residues past the end of the reference protein, as PGAP and Bakta treat a frameshift, which also covers a frame that runs into the next gene
  • Genes a contig end cuts: in a draft assembly they came out as pseudogenes, or as complete genes, without a word. A gene whose frame runs off the contig, or that opens at its edge without a start codon, is now marked partial at that end. In CDC1551 cut into 40 contigs, the larger piece of each of the 25 genes a cut runs through is marked so
  • The other pieces of a cut gene: miniprot reports only the placements within 5% of a protein's best, so only the larger piece of a gene a contig end cuts was annotated. What that placement lacks is now aligned again as a protein of its own, and its placement at the end of another contig is annotated under the gene's name and marked partial: 47 of the 50 pieces in the CDC1551 draft instead of 25, the other three being five or six codons long. A gene that opens at a contig's edge with a codon that can also start one, as one in six codons inside H37Rv genes can, is marked partial too when its protein begins inside its reference protein
  • Disrupted copies of a gene: deduplication kept further copies only when intact, so a whole copy that is a pseudogene was deleted with the pieces of split genes; it now stays when nearly as long as the reference protein
  • GFF3 references: each CDS line was translated on its own, so a CDS of several lines came out in fragments, and proteins under 11 residues were dropped to hide them. The lines are joined per CDS and translated with the bacterial code, reading transl_except, and E. coli K-12's GFF3 gives the same 4318 proteins as its GenBank record, 24 of which were missing
  • Genes longer than their reference keep its name: a Bakta call that won its locus over a shorter alignment in its frame kept no name, and the protein was missing from the table: Rv2024c in CDC1551 and M. bovis, mmpL6 (Rv1557, whole in M. bovis where TbD1 cut it in H37Rv), Rv1068c, Rv1917c and Rv3135 in M. bovis, fliD (b1923) and b0282 in Sakai. The call now takes the name when the alignment spans at least half of the protein, so a remnant does not name the gene it ends (the 67 residues of cyp141 that RD12 leaves in M. bovis)
  • Small proteins with a few substitutions: only exact copies were looked for, so Sakai's ten K-12 micro-proteins with one to four residues changed were not annotated. A protein of up to 50 residues with no exact copy is now looked for with up to a fifth of its residues changed; a stop codon in place of a residue counts, and the gene is then a pseudogene (b1229)
  • Synteny no longer hangs on the order of the rows: the anchors were sorted by start and searched by midpoint, so two anchors starting together sent a placement to whichever window the row order gave, and adding unrelated rows moved Sakai's b2127 24 kb and handed its locus to the paralog b1162. A locus several proteins are placed at exactly no longer anchors either: the single IS6110 of M. bovis made 34 anchors with reference positions all over the genome, and came out under all 17 IS6110 names of each of its genes; they are now Rv2814c and Rv2815c, its own
  • Genes without a COG category are counted once: count_annot.py filled in a missing category as Unassigned and split it into letters, as it splits JK into J and K, so every gene eggNOG gave no category was counted once in A, D, E, G, I and U and twice in N and S, in *-AMAP.COG.tsv and in the report's COG figures and statistics, where it also shrank every other category's share. A test-profile run with a stand-in for eggNOG that gave none of its 4036 genes a category reported 4036 genes in each of those six and 8072 in N and S. Such a gene now counts once, as NaN, which the report shows as None, and its percentage is of all the genome's genes like every other category's
  • COG categories R, W, X, Y and Z in the report: its COG figures took their categories from a list that ran from A to V without R, so a gene eggNOG put in R, W, X, Y or Z was plotted as NA in the overview and in none of the per-category figures, which stayed empty for those five, X - Mobilome (prophages and transposons) among them. The categories now come from the report's own list of figures, A to Z and None, so each is plotted in its own. The statistics table never used that list and is unchanged
  • The QC table counts the pseudogenes AMAP called: annotation_qc.py read the first of Is_Pseudogene, pseudogene and is_pseudogene that the master table has, and every master table has Mycobrowser's Is_Pseudogene, empty when Mycobrowser is off. So pseudogene_count in *-AMAP.qc.tsv and the report's QC table was 0 in every organism-agnostic run (E. coli Sakai against K-12, where AMAP called 129 pseudogenes), and in an MTBC run the number of the sample's genes named after one of the 13 that Mycobrowser marks as pseudogenes in H37Rv, whatever AMAP made of them: 13 for CDC1551, where AMAP calls 123 and finds 6 of those 13 intact. It now counts AMAP's own call, the master table's pseudogene column, the same genes the GFF3 marks pseudo=true and the GenBank /pseudo
  • The report's pseudogene chart counts AMAP's calls, sample by sample: its Pseudogene Detection section counted the master table's genes flagged in Mycobrowser's Is_Pseudogene or in AMAP's pseudogene column, lineage by lineage, as "Pseudogenes per Lineage (Mycobrowser annotation)". Is_Pseudogene says which genes are pseudogenes in H37Rv, not in the sample, so CDC1551 came out at 129 where AMAP calls 123 and the QC table says 123, the other 6 being genes Mycobrowser marks that CDC1551 has intact (Rv0947c, Rv1150, Rv2023A, Rv2098c, Rv3021c, Rv3324A), and without Mycobrowser the chart showed AMAP's calls under Mycobrowser's name. It also left out a lineage with none, and added up a lineage's samples, so a lineage with more samples looked as if it had more pseudogenes. REPORT_INPUTS now hands the report every master table row with the sample it is from, and the chart shows how many pseudogenes AMAP called in each sample, by lineage, a sample with none at 0, next to a table of them. The master table explorer shows the note that says why a gene is a pseudogene
  • A sample with no gene in a category counts 0 in the report: count_annot.py writes a row for each COG or Mycobrowser category a sample has genes in, and the report plotted the rows as they came, so a sample with no gene in a category was left out of that category's figures, of the Dunn tests between lineages run on them, of the overview and of the statistics table, instead of counting 0. Its lineage's box, mean and median were those of the samples that have some, a lineage none of whose samples has any dropped out of the comparison, and the comparison with the median never showed a sample that had lost a category. Each sample now counts 0 in every category another sample of the run has genes in, and the statistics table's Count is every sample of the lineage. A category no sample has keeps its empty figure, and *-AMAP.COG.tsv and *-AMAP.MycoBrowser.tsv are unchanged
  • The clean proteome leaves out the pseudogenes AMAP called: export_clean_proteome.py took them from the first of Is_Pseudogene, pseudogene and is_pseudogene that the master table has, which is Mycobrowser's, and looked each protein up by its whole FASTA ID, L4_CDC1551_Rv0001;+;0-1524 where the master table has L4_CDC1551_Rv0001, so it never found one and its log said 0 pseudogenes. Only the proteins with an internal stop codon were left out, and the pseudogenes without one stayed in *-AMAP.clean.faa, which the docs say has none: 42 of the 123 AMAP calls in CDC1551 (a frameshift or lost stop codon, no stop codon within reach, no stop codon as in the reference) and 27 of the 129 in E. coli Sakai against K-12. It now leaves out the genes whose pseudogene is yes, found by the ID before the ;, and on both genomes holds the CDS the GFF3 does not mark pseudo=true, 3891 and 5194, with the proteins the GenBank translates for them. A master table without that column stops it, where a warning let every pseudogene through
-

๐Ÿ› Pipeline#

-
  • Pseudogenes in the GFF3 and GenBank: the GFF3 never said which genes were pseudogenes, and the GenBank only for genes with an internal stop, though the master table knew. Both now carry AMAP's flag with its reason (pseudo=true and Note, /pseudo and /note), and mark the ends a contig cut off (partial=true with start_range or end_range, < or >); the master table and *-AMAP.tsv have them as their last columns
  • The default H37Rv reference under Docker: its files live inside the container and were handed to processes as paths, so Nextflow mounted /reference from the host, which Docker Desktop refuses and Docker on Linux mounts empty over the container's own. BUNDLED_REFERENCE copies them out first
  • -profile test: VALIDATE_INPUTS still required the sample sheet the test profile does without. 1.0.1 had only been checked with -preview, which runs no task
  • -resume kept results made by older code: Nextflow hashes a task's inputs and script, not the files it runs by path, so after an edit to the classifier or any script under bin/, -resume reused the old results. Every process that runs AMAP code now lists those files in ext code, which puts them in its cache key; edit bin/classify/cases.py and TYPE_CASES and FIX_CDS run again while PATHOTYPR and Bakta stay cached
  • PATHOTYPR no longer depends on the classifier: its script said pathotypr classify, and Nextflow adds any bin/ entry a script names as a bare word to the cache key, so a change in bin/classify/ re-ran PATHOTYPR, Bakta and everything after them. The subcommand is quoted
  • Nextflow 25.10 and later: the pipeline stopped before running anything, because the strict parser those versions use by default rejects the helper functions in nextflow.config, the statements outside the workflow in main.nf, the bare env(SAMPLE_NAME) output and publishDir paths that name a task input. Nextflow 26.10 also refuses the return that ended a --validate_only run. The resource caps are now the resourceLimits directive, which still takes --max_cpus, --max_memory and --max_time from the command line, a profile or a -c config. Nextflow 24.04, where that directive first appeared, is the new minimum
  • --skip_mycobrowser in organism-agnostic mode: a custom reference from another genus turned it on by reassigning the parameter, which current Nextflow ignores with a warning, so the run went ahead with Mycobrowser enabled. It is now worked out in the parameter's default
  • Tasks leave the pipeline untouched: the first time a copy of AMAP ran, Python cached the bytecode of AMAP's own modules inside it, in bin/classify/__pycache__, and with pathotypr classify naming that directory in PATHOTYPR's script, the first -resume re-ran PATHOTYPR, Bakta and everything after them. Tasks now run with PYTHONDONTWRITEBYTECODE=1
  • Samples with no Bakta call outside cases 2 and 3: when every Bakta call had an alignment at exactly its coordinates, which a small input can do, TYPE_CASES stopped with a KeyError, because the empty column of overlaps was read as a list of column names. Case 1 and case 4 now come out empty and the rest is classified as usual. An empty miniprot table stopped it the same way
  • Samples Bakta calls no CDS on: with no contig names from Bakta to compare, the contig check took them for different names and stopped the run, and a Bakta table with nothing but its header did not read. The genes miniprot places now come out as missing from Bakta, like any gene it did not call
  • --version: --help and the parameter docs have listed it since 1.0.0, but nothing read it, so nextflow run PathoGenOmics-Lab/AMAP --version started a run. It now prints the pipeline and Nextflow versions and exits before any task
  • Drafts named after one of their contigs: pathotypr writes a summary row for each contig that carries lineage markers, and PATHOTYPR took the sample's name from the first, so CDC1551 cut into 40 contigs, sample CDC1551d, came out as L4_contig_14, and so did every file after it. Two drafts whose first row named the same contig became one sample, the results of one overwriting the other's. A genome is now named <lineage>_<sampleId> from the sample sheet, the lineage of a draft being the one the markers of all its contigs add up to, and the contigs of a draft keep their names instead of the first one taking the sample's. A closed genome whose header is its sample ID comes out as before
  • Headers with a description: the name was the whole header line, so the tutorial's CDC1551 (NC_002755.2 Mycobacterium tuberculosis CDC1551, complete sequence) was named with spaces in it, Bakta failed on it three times and the run finished without it, and without failing. Such a genome now takes its sample ID like any other, and so does the test profile's: L4_H37Rv_test, where it was L4_CP110619_H37Rv
  • An input could be emptied: PATHOTYPR wrote the renamed genome next to the link Nextflow stages its input as, so when the new name was the file's own, sample G26689 in L4_G26689.fasta for instance, the input was overwritten with nothing and the run went on without it. The input is now staged in a directory of its own
  • Sample IDs with underscores: the steps after the correction told a sample's lineage from its sample ID by splitting its name at underscores, though typing only puts the lineage in front and sample IDs may hold underscores of their own. L4_H37Rv_test was counted as genome H37Rv, the tutorial's Beijing_1 and a Beijing_2 as one genome Beijing in the category counts, the report and the duplication network, an untyped H37Rv_test as lineage H37Rv, and each gene of an untyped sample with no underscore as a genome of its own in the network. PATHOTYPR now says which lineage it named a genome after and every step is handed the sample ID and the lineage. The case summary gains a Sample column and names an untyped sample as it is, not Untyped_<sampleId>, and the report no longer reads the first gene not in the reference as a header, nor stops when every gene has one
  • Genomes kept outside the run's folder: VALIDATE_INPUTS staged only the sample sheet and opened the paths written in it from inside its container, which sees only the folder that the work directory and the sheet share. Genomes anywhere else, /data/genomes while running from ~/project for instance, were "FASTA file not found" and the run stopped before anything ran, and a relative path, which the sheet may use, was read from the task directory with or without a container. The genomes are now inputs of the process, each in a directory of its own so that two with the same file name do not collide, and each is read where Nextflow staged it
  • Runs with functional annotation stopped at validation: VALIDATE_INPUTS looked for --eggnog_db in its own task directory, where the default eggnog_db, relative to the launch directory, never is, and took a database not built yet for an error, though EGGNOG_DB builds it. It now looks where EGGNOG_DB builds it, staged when it exists, and stops the run only when eggnog_db.ok is there without a complete database, since Nextflow then never runs EGGNOG_DB again
  • A Bakta database on the host: --bakta_db reached the tasks as a path the container does not mount. With -profile docker, validation called the database missing and stopped the run, and where nothing validated it, as under -profile test, every sample failed three times and was ignored, and the run ended with nothing annotated and exit 0. A database on the host is now staged into BAKTA_ANNOTATOR, PREPARE_REFERENCE and VALIDATE_INPUTS like any other input; the default inside the container is used as before
  • A warning on every run: validation expected a db.sqlite3 that no Bakta database has. It now checks for version.json and bakta.db, which Bakta cannot run without, so their absence stops the run, and names the database's type and version
  • --strict_validation: the documentation said validation warnings could be made to stop the run, but nothing passed the script's --strict on. The new parameter does
  • eggNOG's database under Docker: EGGNOG_DB built it in a folder it worked out for itself from --eggnog_db, and having no inputs, its container saw that folder only when Nextflow happened to mount it with the work directory and the pipeline's bin/, through a parent folder they share. Otherwise, as when running from /data/project with the pipeline in ~/.nextflow, the database was downloaded and built inside the container and lost with it, and only eggnog_db.ok reached the host: later runs skipped EGGNOG_DB, and ANNOT_EGGNOGMAP_BATCH found no bacteria.dmnd. A database already there without eggnog_db.ok was built again. main.nf now makes the folder on the host and hands it to EGGNOG_DB as an input. A folder left with only eggnog_db.ok needs that file deleted, as validation says
  • eggNOG's database in a folder of any name: ANNOT_EGGNOGMAP_BATCH asked eggNOG-mapper for eggnog_db/bacteria.dmnd and --data_dir eggnog_db, but Nextflow stages a folder under its own name, so with a --eggnog_db called anything else, as the HPC guide's /shared/databases/eggnog or the tutorial's ~/databases/eggnog, EGGNOG_DB built the database and the annotation stopped at "DIAMOND database eggnog_db/bacteria.dmnd not present", with or without a container. The folder is now staged under that one name, whatever it is called. The task's tag and first log line also gave a single sample's FAA size in bytes as the number of samples (batch_1519647_samples under -profile test)
  • Reference files on the host: BAKTA_ANNOTATOR gave Bakta --ref_annotation, or --bakta_proteins, as a path written into its script, which a container does not see, nor a task directory when it is relative, so the organism-agnostic mode failed under -profile docker: with K-12 as the reference for Sakai, Bakta called the "user proteins file" not valid three times, the sample was ignored and the run ended with exit 0. Likewise --pathotyper_snps and --pathotyper_ref on the host left every genome untyped, as NA_<sampleId>, --mycobrowser_ref left the Mycobrowser categories out, and one --miniprot_ref_* on the host while the others kept their defaults made BUNDLED_REFERENCE fail, so nothing was corrected. A file on the host is now staged into the tasks that read it, each in a directory of its own, and a default is read inside the container as before
  • Reference files that are nowhere: a typo in one of those paths, or a default on a run without the container, went unnoticed until the samples were ignored or left untyped. Validation now checks each reference file the run reads and that is not on the host, inside the container where the tasks read it, and stops the run when it is not there. A missing Mycobrowser table only warns, since GET_OUTPUTS goes on without it
  • The pangenome without lineage typing: PANGENE aligned each sample's proteins onto the genomes PATHOTYPR renamed, so with --skip_typing, which every custom reference from another genus needs, it had none and wrote empty files for every sample. It now aligns them onto the sample sheet's genomes, each copied to <sampleId>.fasta first, so that two files of the same name (assembly.fasta from two assemblies) stay apart and a .fna is not passed over
  • Genes not in the reference, sample by sample: the report counted only the samples that had some, so a sample whose genes all matched the reference dropped out of its lineage's box instead of counting 0, and when no sample had any (H37Rv against itself, as in -profile test) the section was an empty plot. Every sample is now counted, the section says so when none has any, and each point is drawn at its count, on an axis from 0, where it was jittered up and down by as much as 0.4
  • The report's pangenome section lost a genome: pangene names each genome <name>#0, and the report read the merged matrix with R's default comment character, so its header stopped at the first #. One genome column was dropped and the gene names became row names, so the core/accessory and frequency plots used a single column and the heatmap never appeared. The missing-gene and duplication tables, which carry the reference annotation's gene IDs as given, are now read without a comment character too
  • Genes across the origin of a circular sequence: Bakta reads a sequence as a complete replicon when its FASTA description says complete (NCBI's "complete genome" and "complete sequence") or circular=true (Unicycler), joins its ends and calls a gene across its origin from its start to its stop, start after end. TYPE_CASES stopped on such a gene: E. coli O157:H7 Sakai with NCBI's headers, whose stcE runs across the origin of plasmid pO157 (92563..2502), failed three times and was left out, and the run finished without failing. Past it, check_cds.py dropped such a gene as invalid coordinates, deduplication and the QC table gave it a negative length, the GFF3 wrote it backwards, the GenBank writer stopped on it and the Panaroo GFF3 spanned it over the whole sequence. It now goes through whole and is written as Bakta writes it: start after end in the tables, join() across the origin in a circular GenBank record, and in the GFF3 its end plus the sequence's length on a region marked Is_circular=true. A gene near the origin of such a sequence is read across it too, so Rop on plasmid pOSAK1, of which Bakta called the first 141 bp, comes out whole. Annotated against K-12, Sakai with NCBI's headers now has 5331 genes in common with the same genome under bare names, which Bakta reads as linear, and differs in stcE and Rop, whole where the linear reading has 804 of stcE's 886 residues and no Rop, and in two phage genes Bakta starts elsewhere on a complete genome
  • Typed closed genomes were annotated as linear: PATHOTYPR renamed a genome of one sequence to <lineage>_<sampleId> and nothing else, so the description Bakta reads a complete, circular replicon from never reached it, whatever the header said, and a gene across the origin of a closed genome came out as two pieces under one name, marked 5' and 3' partial. Only the sequence's ID is replaced now, and the description after it stays, as with --skip_typing. Typed from NCBI's headers, CDC1551, CP110619, AL123456.3 and M. bovis get the same genes as before, in place, name, flags and protein, though Bakta starts a few of its own calls elsewhere on a complete genome (dnaA at 1 instead of 34), so the gene IDs that are Bakta locus tags can be numbered differently. H37Rv turned so that the origin cuts gyrB, gyrA or pknB gives back all 4031 GenBank CDS, where it gave 4030 with the cut gene in two pieces. The GenBank record of such a genome is marked circular, with the description in its definition, and its GFF3 has a region marked Is_circular=true
  • The report's duplication summary: it printed the number of homologous community pairs as markdown from a chunk that did not pass its output on as markdown, so the report showed ## **48** homologous community pairs detected across all genomes. in a code box. It is now a sentence. The example report's duplication table had columns GET_ALLDUP never writes, so its Network Metrics tab said there were none; the demo data now has the columns GET_ALLDUP writes and the example is rendered again
  • Modularity in homologous_communities.tsv: Modularity1 and Modularity2 were computed with the whole community as its only group, which makes them 0 whatever the community, as on all 29 rows of CDC1551 against M. bovis. The two columns are gone
  • COG categories in homologous_communities.tsv: with numpy 2, as in the container, Elements1 and Elements2 read {'L': np.int64(4)}, and they counted each protein eggNOG gave no category in category + or -, since its ID keeps the strand FIX_CDS wrote. They are now plain counts, the largest first and ties in alphabetical order rather than in an order that changed from run to run, and a protein without a category counts as -, eggNOG's mark for none
  • The order of -profile: from Nextflow 25.10, whose strict parser applies the profiles in the order -profile names them and lets the last one win, the laptop caps of the container profiles took over from the others: -profile slurm,singularity held every task of a cluster run to the launching machine's CPUs and 16 GB, where -profile singularity,slurm gave it 48 CPUs and 128 GB, -profile test,docker ran the test with 16 GB and -profile highmem,slurm with 128 GB. No profile sets a cap now: they are set after the profiles, from whether highmem, test or a scheduler profile is on, so every order gives what Nextflow 24.04 gave. The parameter docs gave 24 CPUs and 50 GB as the defaults, which no run got
  • The duplication files are the same from one run to the next: homologous_communities.tsv listed its pairs of communities in the order of a set, which changes with Python's hash seed from one run to the next, and with it which community of a pair came first and the last digit of AvgClustering, an average over the community's proteins that networkx went through in the order of their set: 0.5833333333333333 or 0.5833333333333334 for the same pair of CDC1551 and M. bovis. The pairs now come in the order of the communities, the sample first by name on the left, and the metrics go through the proteins in alphabetical order. Each family in duplications_*_groups.tsv lists its proteins in alphabetical order, where they too came in the order of a set, and each sample's two network drawings no longer start from a random layout and a random partition, so every file both steps write is the same run after run
  • No web browser: both duplication steps wrote their networks with pyvis' show(), which also opens the file in a web browser, so a run without a container on a desktop opened a browser tab for every network. They are written as before, without being opened
  • A custom reference with a gene across the origin, or with several sequences: PREPARE_REFERENCE named each reference protein after where its pieces lie, so tagA on plasmid pO157 of E. coli O157:H7 Sakai, join(92527..92721,1..2502) in NCBI's GenBank record, became pO157p01_1-92721, the whole plasmid, and lost its stop; from NCBI's GFF3, which writes it 92527..95223, it came out as its first 65 of 898 residues. Bakta's stcE on the reference, 92563..2502, matched no gene and was listed among the false positives, and the calls were compared with the genes of every sequence at once: seven calls on pO157 and two on the chromosome counted as matched by genes of the other at the same numbers, and the GenBank's 1-92721 hid two more. With Sakai as its own reference, the sample with NCBI's headers came out with tagA as 1..2502, 65 codons short and not marked partial (GenBank), or from Bakta's later start (GFF3), and with bare headers the GFF3 lost the 2502 bp piece altogether. A gene across the origin now ends past the sequence's last base, as GFF3 writes it (pO157p01_92527-95223, with its stop), each protein names its sequence (contig=), Bakta keeps the FASTA's names on the reference, and each call is compared with the genes of its own sequence, on both sides of the origin; calls on a sequence the annotation has no gene on are not judged. Both formats now give the same 5155 proteins and the same 260 false positives, and the sample with NCBI's headers gets tagA exactly, 92527..2502. K-12's 142 false positives are unchanged
  • pipeline_info/params.yml is the run's own: with -resume it came back from the cache, so it held the first run's parameters whatever later runs were given, since its script reads params as a whole, which Nextflow 26.10's default parser leaves out of the task's hash. And all but the first parameter fell outside params:, with a stray YAML line at the end, because Nextflow strips a script's indentation once the script is filled in. DUMP_PARAMS now never comes from the cache, and writes each value as a JSON string under params:
  • Bakta's SLURM QoS only on SLURM: Bakta's jobs asked for --qos=medium under every scheduler, though only SLURM's sbatch takes it, and under -profile sge it took the place of the memory request the sge profile gives every task. The slurm profile now sets it for Bakta
  • The Mycobrowser table is checked whenever it is read: validation checked --mycobrowser_ref only with the functional annotation, though the interactive report reads it in every run, so under --skip_functional_annotation a wrong path went unwarned and the report came out without Mycobrowser's categories
  • The duplication networks draw offline: both duplication steps loaded their networks' drawing library from a CDN, so without the internet the pages drew nothing. The library now goes inside each page, about 0.7 MB
  • environment.yml pins eggNOG-mapper 2.1.13, the version the container ships, where it said 2.1.12
  • The pangenome shared no gene between samples: PANGENE aligned each sample's own proteins to the other samples' genomes, ran pangene once per sample and named each column <genome>_<sample>#0, and REPORT_INPUTS merged those matrices on gene names that carry the sample's name and coordinates (A4_Mbovis_Rv0001;+;0-1524, L4_CDC1551_Rv0001;+;0-1524), so no row was ever in two samples and no gene was ever core. With N samples the matrix had N(N-1) columns: on the first 2 Mb of CDC1551, M. bovis and H37Rv the report showed 5379 gene families in 6 samples, all shell, with 200 genes of one genome as its heatmap, and as no genome was searched for its own genes, the interactive report found no gene in one genome alone. PANGENE now uses pangene as it is meant to be used: one set of proteins, the reference proteome and each sample's genes that no reference gene matches, aligned to every genome, the sample's own included, and one graph for the run, N alignments where there were N(N-1). A row is a reference gene (Rv1506c) or, outside the reference, one sample's copy of a gene (L4_CDC1551_01396), and a column a sample, so the same run shows 1810 genes, 1760 of them in all three, RD4 and RD10 missing from M. bovis only, the phiRv1 prophage from CDC1551 only, and three genes of the RvD2 region that CDC1551 and M. bovis carry and H37Rv lacks. Its files are now PANGENE/pangenome.{gfa,bubbles.txt,Rtab,stat}, one set for the run instead of one per sample, the graph statistics, which were never published, among them
-

๐Ÿ“Š Report#

-
  • A report organised by question, results/amap_report.html: next to final_report.html, a new process, AMAP_REPORT, writes one self-contained file from each genome's own outputs: a table of the genomes with the values far from comparable genomes flagged, a map of every reference gene in every genome where regions of difference show as blocks, a lookup of one gene in every genome, the correction from Bakta's calls to AMAP's genes with the genes behind each step, the pseudogenes by lineage, what lies outside the reference, and the functional profile, the categories that differ most between lineages first. It needs only what the correction leaves, so a run with --skip_functional_annotation, which gets no final_report.html, gets this one, and it names the genomes of the sample sheet that produced no annotation. Fifteen MTBC genomes make 0.5 MB (final_report.html is 4.4 MB for twelve); it draws everything with its own JavaScript and needs only Python's standard library to write
-

๐Ÿงน Code#

-
  • reconcile_coordinates is gone: it read reference coordinates from protein identifiers that have not carried them since March 2026, so it changed nothing. The paralog check does its job with the sample's own offsets
-

๐Ÿ“š Documentation#

-
  • A website in plain HTML, drawn with real genes: MkDocs is gone. The front page starts a newcomer in three steps and builds the command for their machine, the pipeline is a metro map whose stations are main.nf's processes, and the four cases, the missing genes and the harder cases (paralogs, small proteins, contig ends, frameshifts, copies) are drawn on genes of CDC1551, M. bovis and E. coli from the benchmark. The tutorial and user guide stay markdown, readable on GitHub, and tools/build_docs.py turns them into pages of the same site, dark by default
  • The site is checked against the code: tools/check_site.py runs every drawn window through bin/, compares the thresholds on the pages with the constants in the code, the metro map with main.nf and every option with the pipeline's parameters, so a change that makes the site wrong fails the docs workflow. tools/figure_data.py redraws the windows from a benchmark run
  • What the docs said that the pipeline does not do: with --skip_functional_annotation a run still writes AMAP/, with the corrected table and sequences, where the outputs page said it did not; no step writes PATHOTYPR/renamed_fastas.tsv; and the 19-residue toxin the algorithm page names is IbsC (b4665), not an ldr toxin
  • A drawing on every page that explains something: the duplication family and communities of CDC1551 and M. bovis, from a real run of both duplication steps; the sample sheet with its tabs and the messages the validator prints; the ranges a genome is validated against; the container and what it holds; a run on a cluster; what happens to a task that fails; the report section by section; the five outcomes and the three inputs of a decision; the command builder on the quickstart. Most are built from the code or the config, so they change when it does
  • The benchmark on the front page: the seven genomes whose answer is known, and how many of their genes AMAP finds, checked against tests/benchmark/README.md. A shared link to any page shows a card drawn from the front page's genes
  • More that the docs got wrong: the duplication page listed ten network metrics, the table has four, and its columns ending in 1 and 2 are the two communities of a pair, not two levels; Nextflow needs Java 17, not 11; --skip_functional_annotation leaves no GFF3 or GenBank, which come after eggNOG; tasks are capped at 16 GB on one machine and at 128 GB under a scheduler profile, not 50 GB; and lineage typing is not skipped on its own for other species, only Mycobrowser is
  • The tutorial's numbers are today's: its case counts and QC line, and their copies on the front page, the outputs page and the custom-reference page, came from runs in July. They now come from the tutorial's own commands run again on 18 October 2026 with --skip_functional_annotation, the QC line from GET_OUTPUTS' own script run on that run's corrected genes: CDC1551 has 4014 genes and 123 pseudogenes as AMAP calls them, where the old line counted Mycobrowser's 13, and Sakai keeps 1387 genes K-12 lacks, not 1356. The tutorial also says where --validate_only writes its verdict and where the number of reference proteins extracted is, as neither reaches the console
  • The README in the site's style: the metro map and the five outcomes in light and dark, the benchmark's scores, and the links a newcomer needs first
  • The README opens with the logo, and its drawing plays: the picture at the top was a still of the front page's drawing, dark whatever GitHub's theme. It now plays the drawing's four steps, in light or dark as the theme is, made by tools/pictures.mjs from the front page's own figure. The README's metro map gains the interactive report's station
  • One DOI: the README, CITATION.cff and every page of the website cited the DOI of one version on Zenodo, and the manifest, main.nf and the interactive report the concept DOI, which always resolves to the latest version. All of them cite the concept DOI now
  • Bakta's options on a SLURM cluster: the HPC page says how to give Bakta a site's options, since for Bakta the QoS the slurm profile sets takes the place of -process.clusterOptions
  • The website is served as it is built: GitHub Pages serves docs/ from main, and by default runs Jekyll over it first, which would have made a page of each markdown file with front matter at the address of the page tools/build_docs.py builds from it. docs/.nojekyll turns Jekyll off, and tools/check_site.py fails without it
-

๐Ÿงช Testing#

-
  • tests/selftest/h37rv_selftest.sh annotates H37Rv with itself inside the container and fails unless every GenBank CDS comes back exactly
  • tests/test_cache_keys.py fails when a process runs AMAP code that is not in its cache key, or when its script names a bin/ entry as a bare word without declaring it
  • tests/benchmark/benchmark.sh scores the correction algorithm on seven genomes whose right answer is known (H37Rv twice, CDC1551 complete and in 40 contigs, M. bovis, E. coli K-12 and Sakai), downloading them and running Bakta and miniprot in the container the first time, and lists every gene that changed since an earlier run. A gene whose sequence changed counts as found when a gene with its name covers half of its own bases there, not half of the stretch it lands on, which an insertion can make far longer (63 kb with a prophage inside for b1004 in Sakai); a gene mostly deleted is not counted. Changed genes found: 50 of 50 in CP110619, 860 of 862 in CDC1551, 1515 of 1515 in M. bovis and 3363 of 3367 in Sakai, and the README says why each of the other six is not
  • tests/test_name_by_lineage.py runs PATHOTYPR's own script, rendered as Nextflow renders it, with a stand-in for pathotypr that writes the summaries pathotypr wrote for CDC1551 complete and in 40 contigs, and fails unless Bakta would read a renamed genome as circular exactly when it reads the original so
  • tests/test_sample_ids.py runs TYPE_CASES, GET_OUTPUTS, GET_COUNT, GET_DUPLICATIONS, REPORT_INPUTS and GET_ALLDUP as Nextflow renders them, chained as the pipeline chains them, for a typed and an untyped run whose sample IDs hold underscores, and then the report's own R code where R is installed, and FINAL_REPORT's own script on a run's tables and on one that leaves every table it can empty
  • tests/test_validate_inputs.py runs VALIDATE_INPUTS's own script, rendered as Nextflow renders it, in a task directory laid out as Nextflow lays one out: genomes in a folder the task cannot see, relative paths, two genomes with the same file name and a sheet of 500 samples; and checks both databases, eggNOG's in each state EGGNOG_DB can leave it in
  • tests/test_eggnog_db.py runs EGGNOG_DB's own script, rendered as Nextflow renders it, with its folder staged as Nextflow stages an input and stand-ins for eggNOG-mapper's download and build: the database is built in that folder whatever it is called, one already there is used as it is, and a short bacteria.dmnd is built again
  • tests/test_eggnog_annotation.py runs ANNOT_EGGNOGMAP_BATCH's own script the same way, with a stand-in for emapper.py that looks for the database as eggNOG-mapper does: the database is found whatever its folder is called, a folder called split, where the script writes each sample's annotations, is left as it was, and the tag and log count a single sample as one
  • tests/test_reference_files.py runs the scripts of BAKTA_ANNOTATOR, PATHOTYPR, GET_OUTPUTS and BUNDLED_REFERENCE, rendered as Nextflow renders them, their own Groovy included, with stand-ins for Bakta and pathotypr and each reference file on the host, under a relative path or inside the container; and fails when a process reads a reference file's param other than as what it falls back on when nothing is staged; and checks the Mycobrowser table whenever a step reads it
  • tests/test_pangene_genomes.py runs PANGENE_GENOME and PANGENE's scripts as Nextflow renders them on real genomes of CDC1551, M. bovis and H37Rv cut around RD4 and RvD2 (tests/data/pangenome/, cut again by its make_fixture.py), with stand-ins for miniprot, pangene, k8 and gfatools that hand names on as the real ones do, and fails unless every genome, the sample's own included, is searched with the same proteins, each column is named after its genome, typed or not, a genome none of them is found in keeps a column of zeros, and a protein two samples share goes in once. Where the real tools are installed, as in the container, RD4 must come out missing from M. bovis only and the three RvD2 genes in CDC1551 and M. bovis only, one row each
  • tests/test_report_tables.py merges the matrix PANGENE writes as REPORT_INPUTS does and checks that the report reads every table a # can reach without a comment character; where R is installed, as in the container, it also runs the report's pangenome code on it and counts the genes all its samples share
  • tests/test_cog_counts.py counts COG categories as count_annot.py does, for genes in none, in two and in lowercase, then runs the steps after the correction as tests/test_sample_ids.py chains them, with eggNOG giving some genes no category, then no gene one, then some samples no gene in a category another sample has genes in, and reads what the report's own overview, per-COG figures and statistics table show where R is installed: each category in a figure of its own, with every sample, at 0 where it has no gene. The Mycobrowser figures, statistics and ratios to the median are read the same way for a sample with no gene in one of their categories
  • tests/test_duplication_report.py runs GET_DUPLICATIONS and GET_ALLDUP as Nextflow renders them, chained as the pipeline chains them, with eggNOG giving one copy of a duplicated gene a category, another none and the last no row at all, and checks the columns and COG counts of homologous_communities.tsv; where R is installed, it knits the report's duplication summary on that table and on the one GET_ALLDUP writes when nothing is duplicated. It also fails when a chunk of the report prints markdown without passing it on as markdown
  • tests/test_profile_caps.py fails when a profile sets a cap, or when the order of -profile changes one: it reads nextflow.config and applies its profiles in every order, as both of Nextflow's parsers do, and with Nextflow on PATH also asks nextflow config
  • tests/test_qc_pseudogenes.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling some genes pseudogenes and Mycobrowser marking another, with Mycobrowser on and off, and checks that *-AMAP.qc.tsv, REPORT_INPUTS and the report's own QC table count the pseudogenes AMAP called, as many as the GFF3 and the GenBank mark
  • tests/test_duplication_reproducible.py runs get_duplications.py and network_genomes.py under three hash seeds, on the Diamond tables of three genomes with a family whose average clustering changes with the order of its proteins, and fails unless every file they write is the same under all three; it also fails when either script opens a web browser, with hits or without, as a stand-in browser that records what it is asked to open tells, and when a network page asks a CDN for its drawing library
  • tests/test_custom_reference.py extracts a reference whose plasmid has a gene across its origin from a GenBank record and from GFF3 written either way, on either strand, compares Bakta's calls with it sequence by sequence, and runs PREPARE_REFERENCE's own script, rendered as Nextflow renders it, with a stand-in for Bakta that names sequences as Bakta does
  • tests/test_amap_report.py writes the report from four genomes of a real run cut to two stretches of the reference (tests/data/report/, rebuilt by its make_fixture.py), where M. bovis lacks RD4 and a contig end cuts dnaE2, plus a gene across the origin of a circular plasmid, and checks that every gene, and every Bakta call AMAP did not keep, comes back out of the page as it went in, with Python and, where node is installed, with the page's own JavaScript, along with the states, the flags, the correction's steps and what the process hands the script
  • tests/test_dump_params.py runs DUMP_PARAMS with Nextflow where it is installed, under both parsers, with values holding quotes, $, backticks, a backslash and a line break, and again with -resume and one value changed; everywhere, it fails if the step can come from the cache
  • tests/test_cluster_options.py reads which profile gives Bakta the QoS, and with Nextflow resolves Bakta's and every task's cluster options for each scheduler, in both orders with a container and under both parsers
  • tests/test_environment_pins.py fails when environment.yml pins a tool at a version docker/Dockerfile does not install
  • tests/test_citation.py fails when anything cites AMAP with a DOI other than the concept DOI of the manifest
  • tests/test_clean_proteome.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling some genes pseudogenes, one with a stop codon in its protein and the others without, and Mycobrowser marking a gene AMAP finds intact, with Mycobrowser on and off, and checks that *-AMAP.clean.faa holds every protein FIX_CDS wrote but the pseudogenes', as it wrote them: the CDS the GFF3 does not mark pseudo=true, with the GenBank's translations
  • tests/test_report_pseudogenes.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling a different number of pseudogenes in each sample, none in one, and Mycobrowser marking a gene that every sample has intact, with Mycobrowser on and off, and checks that REPORT_INPUTS hands over each master table row with its sample; where R is installed, it runs the report's own pseudogene section on what REPORT_INPUTS made and checks that it draws each sample at the number AMAP called, the QC table's, a sample with none at 0, under a title that does not name Mycobrowser
  • CI's Python 3.9 job runs on Ubuntu 24.04: GitHub's ubuntu-latest became Ubuntu 26.04, for which setup-python has no Python 3.9, so the job stopped at setup before running a test. 3.9 is the container's Python, so it now runs on 24.04; 3.11 stays on the latest
  • 753 tests, up from 92
-

[1.0.1] - 2026-07-25#

+

Correctness

#
+
  • Paralog copies no longer take another gene's place: a protein with a paralog was placed at both loci, and which copy survived came down to insertion order. On H37Rv itself that lost Rv1150, Rv1945 and Rv2828c; on M.ย bovis it reported genes the strain lacks (the mce3 operon of RD7, the phiRv2 prophage of RD11, plcBC of RD5) as copies sitting on their paralogs. Each placement is now checked against where its protein belongs, predicted from the sample's own offset to the reference so that indels do not throw it off
  • Strand of a resolved conflict: taken from the protein's first placement anywhere in the genome instead of the one being resolved, so esxM (Rv1792), Rv3466 and two IS6110 copies came out at their own coordinates on the wrong strand; a Bakta call chosen over an alignment also took the alignment's strand
  • Reference pseudogenes keep their extent: PE21 (Rv2099c) and PPE48 (Rv3022c), which the reference annotates without a stop, were extended to the next one, and oxyR' (Rv2427A) and Rv3216 lost the partial final codon the reference gives them
  • The 1.0.1 fix for reference pseudogenes never fired: it looked for reference coordinates in a column that has not carried them since March 2026, and its tests wrote them in by hand. check_cds.py now reads the reference proteome instead
  • Alignments that stop short are completed: miniprot can miss a protein's N-terminus (Rv3611 on CP110619) or leave a repetitive C-terminus unaligned further from its stop than the 300ย bp searched; the reference protein now says how much is missing
  • Ties between copies of a gene go to the one Bakta also called, then to the one where the protein belongs by synteny, not the one that comes first in the genome (E.ย coli K-12 b2635)
  • Draft assemblies: genes were compared on coordinates alone, and every contig starts at 1, so cut into 40 contigs CDC1551 had 87% of its proteins translated from the wrong contig. Every gene is now matched, translated and written on its own contig, and Bakta runs with --keep-contig-headers so that its contig names match the FASTA's
  • Caseย 4 is grouped strand by strand: about 8% of its groups (101 of 1255 on H37Rv) put an alignment on one strand together with a Bakta call on the other
  • A complete alignment is not displaced by a longer call in its frame: such a call ends at the same stop and only starts further upstream (plcD on CDC1551). Nor does a call that spans several reference genes win against them (Rv3022c and Rv3021c, one reading frame in CDC1551)
  • Reference genes are protected again: the check looked for coordinates in identifiers that no longer carry them, so any call of similar length could displace a reference gene (Rv1614)
  • Missing genes are recovered by what covers them: genes stored under Bakta locus tags looked missing by name, so a placement a codon off its gene went back in as a second copy. With the caseย 4 changes, ORFs annotated twice fall from 10ย toย 3 on CDC1551 and from 16ย toย 6 on M.ย bovis, where the reference has none
  • Real copies of a gene are kept: deduplicate_rvs.py deleted every copy of a gene but one, 34 genes on E.ย coli O157:H7 Sakai, which were then missing from the annotation and from the duplication analysis. Further copies get identifiers of their own
  • Genes that lost their stop no longer swallow the next gene: check_cds.py extended a gene without a stop to the next one in its frame, which after an indel is the next gene's, so that gene was annotated a second time in the same frame (M.ย bovis Rv0590 over Rv0590A, Rv3373 over Rv3374, Rv2293c over Rv2292c). Such a gene now keeps its extent as a pseudogene, unless the extension only restores residues its reference protein has. ORFs annotated twice fall from 3ย toย 2 on CDC1551 and from 6ย toย 3 on M.ย bovis, and none of those left comes from an extension
  • Small proteins: miniprot placed none of the 33 proteins of 8ย toย 29ย residues in E.ย coli K-12, and put the toxin IbsC (b4665) only on its near-identical neighbour IbsD. Every protein without an exact placement is now looked for in the six-frame translation of the genome, which takes K-12 against itself from 4252 to all 4286 of its genes, its alternative-start isoforms counted with their gene, and Sakai from 198ย toย 220 of 220
  • Frameshifts and lost stops: a gene was extended to the next stop in its frame however far that was, and came out as a complete gene with a C-terminus its reference lacks; 18 in CDC1551 and 20 in M.ย bovis. It now keeps its extent and is flagged as a pseudogene when that stop lies more than ten residues past the end of the reference protein, as PGAP and Bakta treat a frameshift, which also covers a frame that runs into the next gene
  • Genes a contig end cuts: in a draft assembly they came out as pseudogenes, or as complete genes, without a word. A gene whose frame runs off the contig, or that opens at its edge without a start codon, is now marked partial at that end. In CDC1551 cut into 40 contigs, the larger piece of each of the 25 genes a cut runs through is marked so
  • The other pieces of a cut gene: miniprot reports only the placements within 5% of a protein's best, so only the larger piece of a gene a contig end cuts was annotated. What that placement lacks is now aligned again as a protein of its own, and its placement at the end of another contig is annotated under the gene's name and marked partial: 47 of the 50 pieces in the CDC1551 draft instead of 25, the other three being five or six codons long. A gene that opens at a contig's edge with a codon that can also start one, as one in six codons inside H37Rv genes can, is marked partial too when its protein begins inside its reference protein
  • Disrupted copies of a gene: deduplication kept further copies only when intact, so a whole copy that is a pseudogene was deleted with the pieces of split genes; it now stays when nearly as long as the reference protein
  • GFF3 references: each CDS line was translated on its own, so a CDS of several lines came out in fragments, and proteins under 11ย residues were dropped to hide them. The lines are joined per CDS and translated with the bacterial code, reading transl_except, and E.ย coli K-12's GFF3 gives the same 4318 proteins as its GenBank record, 24 of which were missing
  • Genes longer than their reference keep its name: a Bakta call that won its locus over a shorter alignment in its frame kept no name, and the protein was missing from the table: Rv2024c in CDC1551 and M.ย bovis, mmpL6 (Rv1557, whole in M.ย bovis where TbD1 cut it in H37Rv), Rv1068c, Rv1917c and Rv3135 in M.ย bovis, fliD (b1923) and b0282 in Sakai. The call now takes the name when the alignment spans at least half of the protein, so a remnant does not name the gene it ends (the 67ย residues of cyp141 that RD12 leaves in M.ย bovis)
  • Small proteins with a few substitutions: only exact copies were looked for, so Sakai's ten K-12 micro-proteins with one to four residues changed were not annotated. A protein of up to 50ย residues with no exact copy is now looked for with up to a fifth of its residues changed; a stop codon in place of a residue counts, and the gene is then a pseudogene (b1229)
  • Synteny no longer hangs on the order of the rows: the anchors were sorted by start and searched by midpoint, so two anchors starting together sent a placement to whichever window the row order gave, and adding unrelated rows moved Sakai's b2127 24ย kb and handed its locus to the paralog b1162. A locus several proteins are placed at exactly no longer anchors either: the single IS6110 of M.ย bovis made 34 anchors with reference positions all over the genome, and came out under all 17 IS6110 names of each of its genes; they are now Rv2814c and Rv2815c, its own
  • Genes without a COG category are counted once: count_annot.py filled in a missing category as Unassigned and split it into letters, as it splits JK into J and K, so every gene eggNOG gave no category was counted once in A, D, E, G, I and U and twice in N and S, in *-AMAP.COG.tsv and in the report's COG figures and statistics, where it also shrank every other category's share. A test-profile run with a stand-in for eggNOG that gave none of its 4036 genes a category reported 4036 genes in each of those six and 8072 in N and S. Such a gene now counts once, as NaN, which the report shows as None, and its percentage is of all the genome's genes like every other category's
  • COG categories R, W, X, Y and Z in the report: its COG figures took their categories from a list that ran from A to V without R, so a gene eggNOG put in R, W, X, Y or Z was plotted as NA in the overview and in none of the per-category figures, which stayed empty for those five, X - Mobilome (prophages and transposons) among them. The categories now come from the report's own list of figures, A to Z and None, so each is plotted in its own. The statistics table never used that list and is unchanged
  • The QC table counts the pseudogenes AMAP called: annotation_qc.py read the first of Is_Pseudogene, pseudogene and is_pseudogene that the master table has, and every master table has Mycobrowser's Is_Pseudogene, empty when Mycobrowser is off. So pseudogene_count in *-AMAP.qc.tsv and the report's QC table was 0 in every organism-agnostic run (E.ย coli Sakai against K-12, where AMAP called 129 pseudogenes), and in an MTBC run the number of the sample's genes named after one of the 13 that Mycobrowser marks as pseudogenes in H37Rv, whatever AMAP made of them: 13 for CDC1551, where AMAP calls 123 and finds 6 of those 13 intact. It now counts AMAP's own call, the master table's pseudogene column, the same genes the GFF3 marks pseudo=true and the GenBank /pseudo
  • The report's pseudogene chart counts AMAP's calls, sample by sample: its Pseudogene Detection section counted the master table's genes flagged in Mycobrowser's Is_Pseudogene or in AMAP's pseudogene column, lineage by lineage, as "Pseudogenes per Lineage (Mycobrowser annotation)". Is_Pseudogene says which genes are pseudogenes in H37Rv, not in the sample, so CDC1551 came out at 129 where AMAP calls 123 and the QC table says 123, the other 6 being genes Mycobrowser marks that CDC1551 has intact (Rv0947c, Rv1150, Rv2023A, Rv2098c, Rv3021c, Rv3324A), and without Mycobrowser the chart showed AMAP's calls under Mycobrowser's name. It also left out a lineage with none, and added up a lineage's samples, so a lineage with more samples looked as if it had more pseudogenes. REPORT_INPUTS now hands the report every master table row with the sample it is from, and the chart shows how many pseudogenes AMAP called in each sample, by lineage, a sample with none at 0, next to a table of them. The master table explorer shows the note that says why a gene is a pseudogene
  • A sample with no gene in a category counts 0 in the report: count_annot.py writes a row for each COG or Mycobrowser category a sample has genes in, and the report plotted the rows as they came, so a sample with no gene in a category was left out of that category's figures, of the Dunn tests between lineages run on them, of the overview and of the statistics table, instead of counting 0. Its lineage's box, mean and median were those of the samples that have some, a lineage none of whose samples has any dropped out of the comparison, and the comparison with the median never showed a sample that had lost a category. Each sample now counts 0 in every category another sample of the run has genes in, and the statistics table's Count is every sample of the lineage. A category no sample has keeps its empty figure, and *-AMAP.COG.tsv and *-AMAP.MycoBrowser.tsv are unchanged
  • The clean proteome leaves out the pseudogenes AMAP called: export_clean_proteome.py took them from the first of Is_Pseudogene, pseudogene and is_pseudogene that the master table has, which is Mycobrowser's, and looked each protein up by its whole FASTA ID, L4_CDC1551_Rv0001;+;0-1524 where the master table has L4_CDC1551_Rv0001, so it never found one and its log said 0 pseudogenes. Only the proteins with an internal stop codon were left out, and the pseudogenes without one stayed in *-AMAP.clean.faa, which the docs say has none: 42 of the 123 AMAP calls in CDC1551 (a frameshift or lost stop codon, no stop codon within reach, no stop codon as in the reference) and 27 of the 129 in E.ย coli Sakai against K-12. It now leaves out the genes whose pseudogene is yes, found by the ID before the ;, and on both genomes holds the CDS the GFF3 does not mark pseudo=true, 3891 and 5194, with the proteins the GenBank translates for them. A master table without that column stops it, where a warning let every pseudogene through
+

Pipeline

#
+
  • Pseudogenes in the GFF3 and GenBank: the GFF3 never said which genes were pseudogenes, and the GenBank only for genes with an internal stop, though the master table knew. Both now carry AMAP's flag with its reason (pseudo=true and Note, /pseudo and /note), and mark the ends a contig cut off (partial=true with start_range or end_range, < or >); the master table and *-AMAP.tsv have them as their last columns
  • The default H37Rv reference under Docker: its files live inside the container and were handed to processes as paths, so Nextflow mounted /reference from the host, which Docker Desktop refuses and Docker on Linux mounts empty over the container's own. BUNDLED_REFERENCE copies them out first
  • -profile test: VALIDATE_INPUTS still required the sample sheet the test profile does without. 1.0.1 had only been checked with -preview, which runs no task
  • -resume kept results made by older code: Nextflow hashes a task's inputs and script, not the files it runs by path, so after an edit to the classifier or any script under bin/, -resume reused the old results. Every process that runs AMAP code now lists those files in ext code, which puts them in its cache key; edit bin/classify/cases.py and TYPE_CASES and FIX_CDS run again while PATHOTYPR and Bakta stay cached
  • PATHOTYPR no longer depends on the classifier: its script said pathotypr classify, and Nextflow adds any bin/ entry a script names as a bare word to the cache key, so a change in bin/classify/ re-ran PATHOTYPR, Bakta and everything after them. The subcommand is quoted
  • Nextflowย 25.10 and later: the pipeline stopped before running anything, because the strict parser those versions use by default rejects the helper functions in nextflow.config, the statements outside the workflow in main.nf, the bare env(SAMPLE_NAME) output and publishDir paths that name a task input. Nextflowย 26.10 also refuses the return that ended a --validate_only run. The resource caps are now the resourceLimits directive, which still takes --max_cpus, --max_memory and --max_time from the command line, a profile or a -c config. Nextflowย 24.04, where that directive first appeared, is the new minimum
  • --skip_mycobrowser in organism-agnostic mode: a custom reference from another genus turned it on by reassigning the parameter, which current Nextflow ignores with a warning, so the run went ahead with Mycobrowser enabled. It is now worked out in the parameter's default
  • Tasks leave the pipeline untouched: the first time a copy of AMAP ran, Python cached the bytecode of AMAP's own modules inside it, in bin/classify/__pycache__, and with pathotypr classify naming that directory in PATHOTYPR's script, the first -resume re-ran PATHOTYPR, Bakta and everything after them. Tasks now run with PYTHONDONTWRITEBYTECODE=1
  • Samples with no Bakta call outside cases 2 and 3: when every Bakta call had an alignment at exactly its coordinates, which a small input can do, TYPE_CASES stopped with a KeyError, because the empty column of overlaps was read as a list of column names. Caseย 1 and caseย 4 now come out empty and the rest is classified as usual. An empty miniprot table stopped it the same way
  • Samples Bakta calls no CDS on: with no contig names from Bakta to compare, the contig check took them for different names and stopped the run, and a Bakta table with nothing but its header did not read. The genes miniprot places now come out as missing from Bakta, like any gene it did not call
  • --version: --help and the parameter docs have listed it since 1.0.0, but nothing read it, so nextflow run PathoGenOmics-Lab/AMAP --version started a run. It now prints the pipeline and Nextflow versions and exits before any task
  • Drafts named after one of their contigs: pathotypr writes a summary row for each contig that carries lineage markers, and PATHOTYPR took the sample's name from the first, so CDC1551 cut into 40 contigs, sample CDC1551d, came out as L4_contig_14, and so did every file after it. Two drafts whose first row named the same contig became one sample, the results of one overwriting the other's. A genome is now named <lineage>_<sampleId> from the sample sheet, the lineage of a draft being the one the markers of all its contigs add up to, and the contigs of a draft keep their names instead of the first one taking the sample's. A closed genome whose header is its sample ID comes out as before
  • Headers with a description: the name was the whole header line, so the tutorial's CDC1551 (NC_002755.2 Mycobacterium tuberculosis CDC1551, complete sequence) was named with spaces in it, Bakta failed on it three times and the run finished without it, and without failing. Such a genome now takes its sample ID like any other, and so does the test profile's: L4_H37Rv_test, where it was L4_CP110619_H37Rv
  • An input could be emptied: PATHOTYPR wrote the renamed genome next to the link Nextflow stages its input as, so when the new name was the file's own, sample G26689 in L4_G26689.fasta for instance, the input was overwritten with nothing and the run went on without it. The input is now staged in a directory of its own
  • Sample IDs with underscores: the steps after the correction told a sample's lineage from its sample ID by splitting its name at underscores, though typing only puts the lineage in front and sample IDs may hold underscores of their own. L4_H37Rv_test was counted as genome H37Rv, the tutorial's Beijing_1 and a Beijing_2 as one genome Beijing in the category counts, the report and the duplication network, an untyped H37Rv_test as lineage H37Rv, and each gene of an untyped sample with no underscore as a genome of its own in the network. PATHOTYPR now says which lineage it named a genome after and every step is handed the sample ID and the lineage. The case summary gains a Sample column and names an untyped sample as it is, not Untyped_<sampleId>, and the report no longer reads the first gene not in the reference as a header, nor stops when every gene has one
  • Genomes kept outside the run's folder: VALIDATE_INPUTS staged only the sample sheet and opened the paths written in it from inside its container, which sees only the folder that the work directory and the sheet share. Genomes anywhere else, /data/genomes while running from ~/project for instance, were "FASTA file not found" and the run stopped before anything ran, and a relative path, which the sheet may use, was read from the task directory with or without a container. The genomes are now inputs of the process, each in a directory of its own so that two with the same file name do not collide, and each is read where Nextflow staged it
  • Runs with functional annotation stopped at validation: VALIDATE_INPUTS looked for --eggnog_db in its own task directory, where the default eggnog_db, relative to the launch directory, never is, and took a database not built yet for an error, though EGGNOG_DB builds it. It now looks where EGGNOG_DB builds it, staged when it exists, and stops the run only when eggnog_db.ok is there without a complete database, since Nextflow then never runs EGGNOG_DB again
  • A Bakta database on the host: --bakta_db reached the tasks as a path the container does not mount. With -profile docker, validation called the database missing and stopped the run, and where nothing validated it, as under -profile test, every sample failed three times and was ignored, and the run ended with nothing annotated and exit 0. A database on the host is now staged into BAKTA_ANNOTATOR, PREPARE_REFERENCE and VALIDATE_INPUTS like any other input; the default inside the container is used as before
  • A warning on every run: validation expected a db.sqlite3 that no Bakta database has. It now checks for version.json and bakta.db, which Bakta cannot run without, so their absence stops the run, and names the database's type and version
  • --strict_validation: the documentation said validation warnings could be made to stop the run, but nothing passed the script's --strict on. The new parameter does
  • eggNOG's database under Docker: EGGNOG_DB built it in a folder it worked out for itself from --eggnog_db, and having no inputs, its container saw that folder only when Nextflow happened to mount it with the work directory and the pipeline's bin/, through a parent folder they share. Otherwise, as when running from /data/project with the pipeline in ~/.nextflow, the database was downloaded and built inside the container and lost with it, and only eggnog_db.ok reached the host: later runs skipped EGGNOG_DB, and ANNOT_EGGNOGMAP_BATCH found no bacteria.dmnd. A database already there without eggnog_db.ok was built again. main.nf now makes the folder on the host and hands it to EGGNOG_DB as an input. A folder left with only eggnog_db.ok needs that file deleted, as validation says
  • eggNOG's database in a folder of any name: ANNOT_EGGNOGMAP_BATCH asked eggNOG-mapper for eggnog_db/bacteria.dmnd and --data_dir eggnog_db, but Nextflow stages a folder under its own name, so with a --eggnog_db called anything else, as the HPC guide's /shared/databases/eggnog or the tutorial's ~/databases/eggnog, EGGNOG_DB built the database and the annotation stopped at "DIAMOND database eggnog_db/bacteria.dmnd not present", with or without a container. The folder is now staged under that one name, whatever it is called. The task's tag and first log line also gave a single sample's FAA size in bytes as the number of samples (batch_1519647_samples under -profile test)
  • Reference files on the host: BAKTA_ANNOTATOR gave Bakta --ref_annotation, or --bakta_proteins, as a path written into its script, which a container does not see, nor a task directory when it is relative, so the organism-agnostic mode failed under -profile docker: with K-12 as the reference for Sakai, Bakta called the "user proteins file" not valid three times, the sample was ignored and the run ended with exit 0. Likewise --pathotyper_snps and --pathotyper_ref on the host left every genome untyped, as NA_<sampleId>, --mycobrowser_ref left the Mycobrowser categories out, and one --miniprot_ref_* on the host while the others kept their defaults made BUNDLED_REFERENCE fail, so nothing was corrected. A file on the host is now staged into the tasks that read it, each in a directory of its own, and a default is read inside the container as before
  • Reference files that are nowhere: a typo in one of those paths, or a default on a run without the container, went unnoticed until the samples were ignored or left untyped. Validation now checks each reference file the run reads and that is not on the host, inside the container where the tasks read it, and stops the run when it is not there. A missing Mycobrowser table only warns, since GET_OUTPUTS goes on without it
  • The pangenome without lineage typing: PANGENE aligned each sample's proteins onto the genomes PATHOTYPR renamed, so with --skip_typing, which every custom reference from another genus needs, it had none and wrote empty files for every sample. It now aligns them onto the sample sheet's genomes, each copied to <sampleId>.fasta first, so that two files of the same name (assembly.fasta from two assemblies) stay apart and a .fna is not passed over
  • Genes not in the reference, sample by sample: the report counted only the samples that had some, so a sample whose genes all matched the reference dropped out of its lineage's box instead of counting 0, and when no sample had any (H37Rv against itself, as in -profile test) the section was an empty plot. Every sample is now counted, the section says so when none has any, and each point is drawn at its count, on an axis from 0, where it was jittered up and down by as much as 0.4
  • The report's pangenome section lost a genome: pangene names each genome <name>#0, and the report read the merged matrix with R's default comment character, so its header stopped at the first #. One genome column was dropped and the gene names became row names, so the core/accessory and frequency plots used a single column and the heatmap never appeared. The missing-gene and duplication tables, which carry the reference annotation's gene IDs as given, are now read without a comment character too
  • Genes across the origin of a circular sequence: Bakta reads a sequence as a complete replicon when its FASTA description says complete (NCBI's "complete genome" and "complete sequence") or circular=true (Unicycler), joins its ends and calls a gene across its origin from its start to its stop, start after end. TYPE_CASES stopped on such a gene: E.ย coli O157:H7 Sakai with NCBI's headers, whose stcE runs across the origin of plasmid pO157 (92563..2502), failed three times and was left out, and the run finished without failing. Past it, check_cds.py dropped such a gene as invalid coordinates, deduplication and the QC table gave it a negative length, the GFF3 wrote it backwards, the GenBank writer stopped on it and the Panaroo GFF3 spanned it over the whole sequence. It now goes through whole and is written as Bakta writes it: start after end in the tables, join() across the origin in a circular GenBank record, and in the GFF3 its end plus the sequence's length on a region marked Is_circular=true. A gene near the origin of such a sequence is read across it too, so Rop on plasmid pOSAK1, of which Bakta called the first 141ย bp, comes out whole. Annotated against K-12, Sakai with NCBI's headers now has 5331 genes in common with the same genome under bare names, which Bakta reads as linear, and differs in stcE and Rop, whole where the linear reading has 804 of stcE's 886ย residues and no Rop, and in two phage genes Bakta starts elsewhere on a complete genome
  • Typed closed genomes were annotated as linear: PATHOTYPR renamed a genome of one sequence to <lineage>_<sampleId> and nothing else, so the description Bakta reads a complete, circular replicon from never reached it, whatever the header said, and a gene across the origin of a closed genome came out as two pieces under one name, marked 5' and 3' partial. Only the sequence's ID is replaced now, and the description after it stays, as with --skip_typing. Typed from NCBI's headers, CDC1551, CP110619, AL123456.3 and M.ย bovis get the same genes as before, in place, name, flags and protein, though Bakta starts a few of its own calls elsewhere on a complete genome (dnaA at 1 instead of 34), so the gene IDs that are Bakta locus tags can be numbered differently. H37Rv turned so that the origin cuts gyrB, gyrA or pknB gives back all 4031 GenBank CDS, where it gave 4030 with the cut gene in two pieces. The GenBank record of such a genome is marked circular, with the description in its definition, and its GFF3 has a region marked Is_circular=true
  • The report's duplication summary: it printed the number of homologous community pairs as markdown from a chunk that did not pass its output on as markdown, so the report showed ## **48** homologous community pairs detected across all genomes. in a code box. It is now a sentence. The example report's duplication table had columns GET_ALLDUP never writes, so its Network Metrics tab said there were none; the demo data now has the columns GET_ALLDUP writes and the example is rendered again
  • Modularity in homologous_communities.tsv: Modularity1 and Modularity2 were computed with the whole community as its only group, which makes them 0 whatever the community, as on all 29 rows of CDC1551 against M.ย bovis. The two columns are gone
  • COG categories in homologous_communities.tsv: with numpy 2, as in the container, Elements1 and Elements2 read {'L': np.int64(4)}, and they counted each protein eggNOG gave no category in category + or -, since its ID keeps the strand FIX_CDS wrote. They are now plain counts, the largest first and ties in alphabetical order rather than in an order that changed from run to run, and a protein without a category counts as -, eggNOG's mark for none
  • The order of -profile: from Nextflowย 25.10, whose strict parser applies the profiles in the order -profile names them and lets the last one win, the laptop caps of the container profiles took over from the others: -profile slurm,singularity held every task of a cluster run to the launching machine's CPUs and 16ย GB, where -profile singularity,slurm gave it 48ย CPUs and 128ย GB, -profile test,docker ran the test with 16ย GB and -profile highmem,slurm with 128ย GB. No profile sets a cap now: they are set after the profiles, from whether highmem, test or a scheduler profile is on, so every order gives what Nextflowย 24.04 gave. The parameter docs gave 24ย CPUs and 50ย GB as the defaults, which no run got
  • The duplication files are the same from one run to the next: homologous_communities.tsv listed its pairs of communities in the order of a set, which changes with Python's hash seed from one run to the next, and with it which community of a pair came first and the last digit of AvgClustering, an average over the community's proteins that networkx went through in the order of their set: 0.5833333333333333 or 0.5833333333333334 for the same pair of CDC1551 and M.ย bovis. The pairs now come in the order of the communities, the sample first by name on the left, and the metrics go through the proteins in alphabetical order. Each family in duplications_*_groups.tsv lists its proteins in alphabetical order, where they too came in the order of a set, and each sample's two network drawings no longer start from a random layout and a random partition, so every file both steps write is the same run after run
  • No web browser: both duplication steps wrote their networks with pyvis' show(), which also opens the file in a web browser, so a run without a container on a desktop opened a browser tab for every network. They are written as before, without being opened
  • A custom reference with a gene across the origin, or with several sequences: PREPARE_REFERENCE named each reference protein after where its pieces lie, so tagA on plasmid pO157 of E.ย coli O157:H7 Sakai, join(92527..92721,1..2502) in NCBI's GenBank record, became pO157p01_1-92721, the whole plasmid, and lost its stop; from NCBI's GFF3, which writes it 92527..95223, it came out as its first 65 of 898ย residues. Bakta's stcE on the reference, 92563..2502, matched no gene and was listed among the false positives, and the calls were compared with the genes of every sequence at once: seven calls on pO157 and two on the chromosome counted as matched by genes of the other at the same numbers, and the GenBank's 1-92721 hid two more. With Sakai as its own reference, the sample with NCBI's headers came out with tagA as 1..2502, 65 codons short and not marked partial (GenBank), or from Bakta's later start (GFF3), and with bare headers the GFF3 lost the 2502ย bp piece altogether. A gene across the origin now ends past the sequence's last base, as GFF3 writes it (pO157p01_92527-95223, with its stop), each protein names its sequence (contig=), Bakta keeps the FASTA's names on the reference, and each call is compared with the genes of its own sequence, on both sides of the origin; calls on a sequence the annotation has no gene on are not judged. Both formats now give the same 5155 proteins and the same 260 false positives, and the sample with NCBI's headers gets tagA exactly, 92527..2502. K-12's 142 false positives are unchanged
  • pipeline_info/params.yml is the run's own: with -resume it came back from the cache, so it held the first run's parameters whatever later runs were given, since its script reads params as a whole, which Nextflowย 26.10's default parser leaves out of the task's hash. And all but the first parameter fell outside params:, with a stray YAML line at the end, because Nextflow strips a script's indentation once the script is filled in. DUMP_PARAMS now never comes from the cache, and writes each value as a JSON string under params:
  • Bakta's SLURM QoS only on SLURM: Bakta's jobs asked for --qos=medium under every scheduler, though only SLURM's sbatch takes it, and under -profile sge it took the place of the memory request the sge profile gives every task. The slurm profile now sets it for Bakta
  • The Mycobrowser table is checked whenever it is read: validation checked --mycobrowser_ref only with the functional annotation, though the interactive report reads it in every run, so under --skip_functional_annotation a wrong path went unwarned and the report came out without Mycobrowser's categories
  • The duplication networks draw offline: both duplication steps loaded their networks' drawing library from a CDN, so without the internet the pages drew nothing. The library now goes inside each page, about 0.7ย MB
  • environment.yml pins eggNOG-mapper 2.1.13, the version the container ships, where it said 2.1.12
  • The pangenome shared no gene between samples: PANGENE aligned each sample's own proteins to the other samples' genomes, ran pangene once per sample and named each column <genome>_<sample>#0, and REPORT_INPUTS merged those matrices on gene names that carry the sample's name and coordinates (A4_Mbovis_Rv0001;+;0-1524, L4_CDC1551_Rv0001;+;0-1524), so no row was ever in two samples and no gene was ever core. With N samples the matrix had N(N-1) columns: on the first 2ย Mb of CDC1551, M.ย bovis and H37Rv the report showed 5379 gene families in 6 samples, all shell, with 200 genes of one genome as its heatmap, and as no genome was searched for its own genes, the interactive report found no gene in one genome alone. PANGENE now uses pangene as it is meant to be used: one set of proteins, the reference proteome and each sample's genes that no reference gene matches, aligned to every genome, the sample's own included, and one graph for the run, N alignments where there were N(N-1). A row is a reference gene (Rv1506c) or, outside the reference, one sample's copy of a gene (L4_CDC1551_01396), and a column a sample, so the same run shows 1810 genes, 1760 of them in all three, RD4 and RD10 missing from M.ย bovis only, the phiRv1 prophage from CDC1551 only, and three genes of the RvD2 region that CDC1551 and M.ย bovis carry and H37Rv lacks. Its files are now PANGENE/pangenome.{gfa,bubbles.txt,Rtab,stat}, one set for the run instead of one per sample, the graph statistics, which were never published, among them
+

Report

#
+
  • A report organised by question, results/amap_report.html: next to final_report.html, a new process, AMAP_REPORT, writes one self-contained file from each genome's own outputs: a table of the genomes with the values far from comparable genomes flagged, a map of every reference gene in every genome where regions of difference show as blocks, a lookup of one gene in every genome, the correction from Bakta's calls to AMAP's genes with the genes behind each step, the pseudogenes by lineage, what lies outside the reference, and the functional profile, the categories that differ most between lineages first. It needs only what the correction leaves, so a run with --skip_functional_annotation, which gets no final_report.html, gets this one, and it names the genomes of the sample sheet that produced no annotation. Fifteen MTBC genomes make 0.5ย MB (final_report.html is 4.4ย MB for twelve); it draws everything with its own JavaScript and needs only Python's standard library to write
  • final_report.html opens with no network: every report carried a script that fetched MathJax from mathjax.rstudio.com, for maths it never shows; mathjax: null leaves it out, so the report opens offline, as the example on the website does. It also declares its language, gives its figures alt text and its tab an icon
+

Code

#
+
  • reconcile_coordinates is gone: it read reference coordinates from protein identifiers that have not carried them since March 2026, so it changed nothing. The paralog check does its job with the sample's own offsets
+

Documentation

#
+
  • A website in plain HTML, drawn with real genes: MkDocs is gone. The front page starts a newcomer in four steps and builds the command for their machine, the pipeline is a metro map whose stations are main.nf's processes, and the four cases, the missing genes and the harder cases (paralogs, small proteins, contig ends, frameshifts, copies) are drawn on genes of CDC1551, M.ย bovis and E.ย coli from the benchmark. The tutorial and user guide stay markdown, readable on GitHub, and tools/build_docs.py turns them into pages of the same site, dark by default
  • The site is checked against the code: tools/check_site.py runs every drawn window through bin/, compares the thresholds on the pages with the constants in the code, the metro map with main.nf and every option with the pipeline's parameters, so a change that makes the site wrong fails the docs workflow. tools/figure_data.py redraws the windows from a benchmark run
  • What the docs said that the pipeline does not do: with --skip_functional_annotation a run still writes AMAP/, with the corrected table and sequences, where the outputs page said it did not; no step writes PATHOTYPR/renamed_fastas.tsv; and the 19-residue toxin the algorithm page names is IbsC (b4665), not an ldr toxin
  • A drawing on every page that explains something: the duplication family and communities of CDC1551 and M.ย bovis, from a real run of both duplication steps; the sample sheet with its tabs and the messages the validator prints; the ranges a genome is validated against; the container and what it holds; a run on a cluster; what happens to a task that fails; the report section by section; the five outcomes and the three inputs of a decision; the command builder on the quickstart. Most are built from the code or the config, so they change when it does
  • The benchmark on the front page: the seven genomes whose answer is known, and how many of their genes AMAP finds, checked against tests/benchmark/README.md. A shared link to any page shows a card drawn from the front page's genes
  • More that the docs got wrong: the duplication page listed ten network metrics, the table has four, and its columns ending in 1 and 2 are the two communities of a pair, not two levels; Nextflow needs Javaย 17, not 11; --skip_functional_annotation leaves no GFF3 or GenBank, which come after eggNOG; tasks are capped at 16ย GB on one machine and at 128ย GB under a scheduler profile, not 50ย GB; and lineage typing is not skipped on its own for other species, only Mycobrowser is
  • The tutorial's numbers are today's: its case counts and QC line, and their copies on the front page, the outputs page and the custom-reference page, came from runs in July. They now come from the tutorial's own commands run again on 18 October 2026 with --skip_functional_annotation, the QC line from GET_OUTPUTS' own script run on that run's corrected genes: CDC1551 has 4014 genes and 123 pseudogenes as AMAP calls them, where the old line counted Mycobrowser's 13, and Sakai keeps 1387 genes K-12 lacks, not 1356. The tutorial also says where --validate_only writes its verdict and where the number of reference proteins extracted is, as neither reaches the console
  • The README in the site's style: the metro map and the five outcomes in light and dark, the benchmark's scores, and the links a newcomer needs first
  • The README opens with the logo, and its drawing plays: the picture at the top was a still of the front page's drawing, dark whatever GitHub's theme. It now plays the drawing's four steps, in light or dark as the theme is, made by tools/pictures.mjs from the front page's own figure. The README's metro map gains the interactive report's station
  • One DOI: the README, CITATION.cff and every page of the website cited the DOI of one version on Zenodo, and the manifest, main.nf and the interactive report the concept DOI, which always resolves to the latest version. All of them cite the concept DOI now
  • Bakta's options on a SLURM cluster: the HPC page says how to give Bakta a site's options, since for Bakta the QoS the slurm profile sets takes the place of -process.clusterOptions
  • The website is served as it is built: GitHub Pages serves docs/ from main, and by default runs Jekyll over it first, which would have made a page of each markdown file with front matter at the address of the page tools/build_docs.py builds from it. docs/.nojekyll turns Jekyll off, and tools/check_site.py fails without it
  • Website: find anything, read it at any width, and trust what it says: a search on every page, from the header, / or Ctrl+K, finds any section, option, story, front-page question or changelog entry, and each option has a link of its own. Docs pages read in a wider column with their contents on every screen, code that never breaks inside a word, and a missing page that points to the nearest one. Every drawing works by keyboard, touch and screen reader and holds still while stepped through. The front page shows Bakta alone against AMAP, what a working test prints and how long it takes, and commands true for each engine. The pages say what the pipeline does today, in the same words everywhere, and tools/check_site.py checks those claims
  • Website: the questions a first run raises: search answers questions asked in plain words, Troubleshooting says why a sample can be missing from a run that completed, a failed input check is described as it behaves (its report in the terminal, no validation.log), the laptop advice is the same on every page, and links to headings renamed since, the changelog's GitHub anchors among them, still land
+

Testing

#
+
  • tests/selftest/h37rv_selftest.sh annotates H37Rv with itself inside the container and fails unless every GenBank CDS comes back exactly
  • tests/test_cache_keys.py fails when a process runs AMAP code that is not in its cache key, or when its script names a bin/ entry as a bare word without declaring it
  • tests/benchmark/benchmark.sh scores the correction algorithm on seven genomes whose right answer is known (H37Rv twice, CDC1551 complete and in 40 contigs, M.ย bovis, E.ย coli K-12 and Sakai), downloading them and running Bakta and miniprot in the container the first time, and lists every gene that changed since an earlier run. A gene whose sequence changed counts as found when a gene with its name covers half of its own bases there, not half of the stretch it lands on, which an insertion can make far longer (63ย kb with a prophage inside for b1004 in Sakai); a gene mostly deleted is not counted. Changed genes found: 50 of 50 in CP110619, 860 of 862 in CDC1551, 1515 of 1515 in M.ย bovis and 3363 of 3367 in Sakai, and the README says why each of the other six is not
  • tests/test_name_by_lineage.py runs PATHOTYPR's own script, rendered as Nextflow renders it, with a stand-in for pathotypr that writes the summaries pathotypr wrote for CDC1551 complete and in 40 contigs, and fails unless Bakta would read a renamed genome as circular exactly when it reads the original so
  • tests/test_sample_ids.py runs TYPE_CASES, GET_OUTPUTS, GET_COUNT, GET_DUPLICATIONS, REPORT_INPUTS and GET_ALLDUP as Nextflow renders them, chained as the pipeline chains them, for a typed and an untyped run whose sample IDs hold underscores, and then the report's own R code where R is installed, and FINAL_REPORT's own script on a run's tables and on one that leaves every table it can empty
  • tests/test_validate_inputs.py runs VALIDATE_INPUTS's own script, rendered as Nextflow renders it, in a task directory laid out as Nextflow lays one out: genomes in a folder the task cannot see, relative paths, two genomes with the same file name and a sheet of 500 samples; and checks both databases, eggNOG's in each state EGGNOG_DB can leave it in
  • tests/test_eggnog_db.py runs EGGNOG_DB's own script, rendered as Nextflow renders it, with its folder staged as Nextflow stages an input and stand-ins for eggNOG-mapper's download and build: the database is built in that folder whatever it is called, one already there is used as it is, and a short bacteria.dmnd is built again
  • tests/test_eggnog_annotation.py runs ANNOT_EGGNOGMAP_BATCH's own script the same way, with a stand-in for emapper.py that looks for the database as eggNOG-mapper does: the database is found whatever its folder is called, a folder called split, where the script writes each sample's annotations, is left as it was, and the tag and log count a single sample as one
  • tests/test_reference_files.py runs the scripts of BAKTA_ANNOTATOR, PATHOTYPR, GET_OUTPUTS and BUNDLED_REFERENCE, rendered as Nextflow renders them, their own Groovy included, with stand-ins for Bakta and pathotypr and each reference file on the host, under a relative path or inside the container; and fails when a process reads a reference file's param other than as what it falls back on when nothing is staged; and checks the Mycobrowser table whenever a step reads it
  • tests/test_pangene_genomes.py runs PANGENE_GENOME and PANGENE's scripts as Nextflow renders them on real genomes of CDC1551, M.ย bovis and H37Rv cut around RD4 and RvD2 (tests/data/pangenome/, cut again by its make_fixture.py), with stand-ins for miniprot, pangene, k8 and gfatools that hand names on as the real ones do, and fails unless every genome, the sample's own included, is searched with the same proteins, each column is named after its genome, typed or not, a genome none of them is found in keeps a column of zeros, and a protein two samples share goes in once. Where the real tools are installed, as in the container, RD4 must come out missing from M.ย bovis only and the three RvD2 genes in CDC1551 and M.ย bovis only, one row each
  • tests/test_report_tables.py merges the matrix PANGENE writes as REPORT_INPUTS does and checks that the report reads every table a # can reach without a comment character; where R is installed, as in the container, it also runs the report's pangenome code on it and counts the genes all its samples share
  • tests/test_cog_counts.py counts COG categories as count_annot.py does, for genes in none, in two and in lowercase, then runs the steps after the correction as tests/test_sample_ids.py chains them, with eggNOG giving some genes no category, then no gene one, then some samples no gene in a category another sample has genes in, and reads what the report's own overview, per-COG figures and statistics table show where R is installed: each category in a figure of its own, with every sample, at 0 where it has no gene. The Mycobrowser figures, statistics and ratios to the median are read the same way for a sample with no gene in one of their categories
  • tests/test_duplication_report.py runs GET_DUPLICATIONS and GET_ALLDUP as Nextflow renders them, chained as the pipeline chains them, with eggNOG giving one copy of a duplicated gene a category, another none and the last no row at all, and checks the columns and COG counts of homologous_communities.tsv; where R is installed, it knits the report's duplication summary on that table and on the one GET_ALLDUP writes when nothing is duplicated. It also fails when a chunk of the report prints markdown without passing it on as markdown
  • tests/test_profile_caps.py fails when a profile sets a cap, or when the order of -profile changes one: it reads nextflow.config and applies its profiles in every order, as both of Nextflow's parsers do, and with Nextflow on PATH also asks nextflow config
  • tests/test_qc_pseudogenes.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling some genes pseudogenes and Mycobrowser marking another, with Mycobrowser on and off, and checks that *-AMAP.qc.tsv, REPORT_INPUTS and the report's own QC table count the pseudogenes AMAP called, as many as the GFF3 and the GenBank mark
  • tests/test_duplication_reproducible.py runs get_duplications.py and network_genomes.py under three hash seeds, on the Diamond tables of three genomes with a family whose average clustering changes with the order of its proteins, and fails unless every file they write is the same under all three; it also fails when either script opens a web browser, with hits or without, as a stand-in browser that records what it is asked to open tells, and when a network page asks a CDN for its drawing library
  • tests/test_custom_reference.py extracts a reference whose plasmid has a gene across its origin from a GenBank record and from GFF3 written either way, on either strand, compares Bakta's calls with it sequence by sequence, and runs PREPARE_REFERENCE's own script, rendered as Nextflow renders it, with a stand-in for Bakta that names sequences as Bakta does
  • tests/test_amap_report.py writes the report from four genomes of a real run cut to two stretches of the reference (tests/data/report/, rebuilt by its make_fixture.py), where M.ย bovis lacks RD4 and a contig end cuts dnaE2, plus a gene across the origin of a circular plasmid, and checks that every gene, and every Bakta call AMAP did not keep, comes back out of the page as it went in, with Python and, where node is installed, with the page's own JavaScript, along with the states, the flags, the correction's steps and what the process hands the script
  • tests/test_dump_params.py runs DUMP_PARAMS with Nextflow where it is installed, under both parsers, with values holding quotes, $, backticks, a backslash and a line break, and again with -resume and one value changed; everywhere, it fails if the step can come from the cache
  • tests/test_cluster_options.py reads which profile gives Bakta the QoS, and with Nextflow resolves Bakta's and every task's cluster options for each scheduler, in both orders with a container and under both parsers
  • tests/test_environment_pins.py fails when environment.yml pins a tool at a version docker/Dockerfile does not install
  • tests/test_citation.py fails when anything cites AMAP with a DOI other than the concept DOI of the manifest
  • tests/test_clean_proteome.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling some genes pseudogenes, one with a stop codon in its protein and the others without, and Mycobrowser marking a gene AMAP finds intact, with Mycobrowser on and off, and checks that *-AMAP.clean.faa holds every protein FIX_CDS wrote but the pseudogenes', as it wrote them: the CDS the GFF3 does not mark pseudo=true, with the GenBank's translations
  • tests/test_report_pseudogenes.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling a different number of pseudogenes in each sample, none in one, and Mycobrowser marking a gene that every sample has intact, with Mycobrowser on and off, and checks that REPORT_INPUTS hands over each master table row with its sample; where R is installed, it runs the report's own pseudogene section on what REPORT_INPUTS made and checks that it draws each sample at the number AMAP called, the QC table's, a sample with none at 0, under a title that does not name Mycobrowser
  • CI's Python 3.9 job runs on Ubuntu 24.04: GitHub's ubuntu-latest became Ubuntu 26.04, for which setup-python has no Python 3.9, so the job stopped at setup before running a test. 3.9 is the container's Python, so it now runs on 24.04; 3.11 stays on the latest
  • The benchmark scores Bakta alone: tests/benchmark/benchmark.sh scores Bakta's own calls of each genome by the same rules as AMAP's table, in two more columns of its README. Bakta alone finds 67ย toย 73% of the unchanged genes at exactly their coordinates and under their name (K-12, 90%), where AMAP finds all of them
  • 753 tests, up from 92
+

1.0.1, 25 July 2026

#

Bug fixes, most of them found by running the pipeline on real genomes rather than by reading it. Several changed what the pipeline outputs, so results from 1.0.0 are not directly comparable.

-

๐Ÿ”ฌ Correctness#

-
  • Genes no longer vanish during overlap resolution: _insert_with_dedup queued entries for eviction while scanning for conflicts, and applied them even when it then abandoned the insert, so genes were deleted with nothing put in their place
  • Reference pseudogenes are recognised for any organism: the check was gated on the identifier starting with Rv, so on every other organism every reference pseudogene was extended to the next stop and lost its flag. The same gene came out differently depending on what it was called
  • The extracted reference proteome carries its stop codon: extract_ref_proteins.py wrote coordinates spanning the stop but a protein without it, so miniprot's alignment ended three bases short of every Bakta CDS and no gene could ever reach case 2. On E. coli O157:H7 this moved 3325 genes out of conflict resolution
  • Mycobrowser join key: built with rvs.split('_')[0], which returned the whole string for a gene with several references (Rv0094c, Rv3467) and truncated any name containing an underscore (PE_PGRS1 became PE). 44 more genes get a functional category on a real MTBC sample, none lose one
  • Annotation table and FASTA files agree: check_cds.py wrote the FASTA files from dictionaries filled before its final deduplication, and deduplicate_rvs.py matched FASTA headers on the gene id alone, so removing one copy of a multi-copy gene deleted all of them
  • Untranslatable genes are dropped from the table, not left in it with no sequence anywhere
-

๐Ÿ› Crashes#

-
  • --skip_mycobrowser and organism-agnostic mode: the placeholder frame was built with a duplicate Locus column, so pd.merge refused to run and GET_OUTPUTS failed on every run without Mycobrowser
  • Miniprot PAF parsing: pandas inferred the column count from the first row, and miniprot varies its trailing tags per alignment, so any file whose first row was narrower aborted the parse
  • Biopython 1.85 and newer: SeqFeature.strand was removed; to_panaroo.py and extract_ref_proteins.py now read it off the location
  • -profile test: pointed at a sample sheet naming paths that are not in this repository. It now annotates the reference genome bundled in the container, so it needs no data and nothing downloaded
-

๐Ÿ“‰ Silent data loss#

-
  • First Diamond hit: read_and_create left header inference on while stripping # comments, so the first alignment became the column names and disappeared from the duplication analysis
  • Multi-part gene names in rename_fastas.py: headers were cut at the third underscore, collapsing every PE/PPE gene of a sample onto one identifier
  • eggNOG annotation rate: counted with notna(), and emapper writes - for fields it could not fill, so the QC metric read close to 100% whatever the run had done
-

โšก Performance#

-
  • reconcile_coordinates is linear rather than quadratic: it scanned every protein and every reference for each gene, calling a regex on each candidate. Indexed by gene name up front, a 4000-gene genome goes from 18.3 s to 0.011 s with identical output over 200 randomised genomes
  • deduplicate: list.count() per element and a full scan per duplicated gene, replaced with a Counter and a single grouping pass
-

๐Ÿ“Š Report#

-
  • Layout: the stylesheet was losing to the one rmarkdown inlines afterwards, so the article sat underneath the table of contents and the left logo on top of the title
  • Self-contained without a network at render time: jQuery came from a CDN and the second logo from GitHub. pandoc inlines those when the render machine has a network and leaves them dangling when it does not, so a report rendered on an isolated node arrived with a broken image
  • Half the size: 8.2 MB to 4.4 MB, by shipping only the plotly traces actually used and quantising the figures
  • The version in the report is the version that produced it: it was written by hand in three places in the Rmd, so any release that forgot to update them published a report naming the wrong version. It now comes from the manifest in nextflow.config
-

๐Ÿ“š Documentation and infrastructure#

-
  • Documentation site built with MkDocs Material, published to GitHub Pages, including a tutorial with the numbers from real runs and a rendered example report
  • Docker image workflow: builds, smoke tests and publishes on a v* tag or on demand
  • Tests workflow enabled: it had been sitting disabled and would have failed, since its dependency list was missing plotly and its syntax check had drifted five files behind
  • requirements-test.txt: one dependency list for CI, the docs and a local checkout. environment.yml was missing the Python plotly, so a conda environment built from it could not start classify_cases.py
  • 92 tests, up from 58
-

[1.0.0] - 2026-04-01#

-

๐Ÿ”ฌ Correctness#

-
  • Strand-aware gene dictionary: combined_dict keys changed from (Start, End) to (Start, End, Strand): genes on opposite strands at the same coordinates no longer collide silently
  • Consistent 1-based lengths: all length calculations now use End - Start + 1 (15 sites fixed)
  • Robust ID cleaning: regex _\d+-\d+$ replaces brittle 4-part underscore rule, so PE_PGRS genes are handled correctly
  • Exact gene name matching: prevents false matches (Rv3508 โ‰  Rv3508c, Rv1 โ‰  Rv10)
  • IntervalTree half-open intervals: all 9 operations corrected from [Start:End] to [Start:End+1]
  • Cross-strand gene preservation: merge operations now include Strand to prevent loss
  • Paralog preservation: multi-copy PE/PPE genes no longer collapsed during groupby
  • Pseudogene propagation: Is_Pseudogene column now reaches master table and report
-

๐Ÿ›ก๏ธ Robustness#

-
  • Pre-flight validation: new VALIDATE_INPUTS process checks FASTA format, N-content, GC%, genome size, duplicate headers and database paths, before any computation
  • --validate_only mode: check all inputs without executing the pipeline
  • eggNOG DB integrity: bacteria.dmnd <100 MB detected as corrupted โ†’ automatic rebuild
  • Bakta DB pre-check: actionable error message with download command
  • OOM retry: memory scales with task.attempt (e.g., 50 GB โ†’ 100 GB โ†’ 150 GB)
  • 0-protein handling: eggNOG batch handles empty inputs gracefully
  • Completion handler: prints duration, succeeded/failed/ignored counts
-

๐Ÿ“‹ Reproducibility#

-
  • nextflow.manifest: version 1.0.0, Nextflow โ‰ฅ23.04.0, DOI, homepage
  • SOFTWARE_VERSIONS: YAML + TSV with exact versions of 12 tools
  • DUMP_PARAMS: complete parameter dump per run
  • Diamond sorted output: deterministic results regardless of thread count
  • environment.yml: Conda environment with pinned versions
  • Louvain seed: fixed to 42 for reproducible community detection
-

โšก Performance & HPC#

-
  • Dynamic resource scaling: memory grows with each retry attempt
  • Node-local scratch: --scratch true (default on SLURM/SGE/PBS/LSF)
  • Diamond block size tuning: auto-calculated from available RAM
  • Bakta local temp: --tmp-dir on node-local disk (avoids NFS metadata storms)
  • eggNOG batch mode: single run for all samples (10ร— faster, 50ร— less filesystem I/O)
  • Thread utilization: Bakta 12, Diamond 6-24, Miniprot 2ร—3, Pangene 2ร—N
  • SLURM/SGE/PBS/LSF profiles: with time limits, queue sizes, rate limiting
  • storeDir for eggNOG DB: persists across runs, survives nextflow clean
-

๐Ÿ“ Scalability#

-
  • find -print0 | xargs -0: replaces echo|tr|grep in REPORT_INPUTS (handles 500+ files)
  • publishDir mode: 'link': hardlinks instead of copies for per-sample outputs
  • cache 'lenient': tolerates NFS timestamp drift between cluster nodes
  • awk close(): prevents file handle exhaustion on 500+ sample eggNOG splits
  • JVM tuning docs: NXF_OPTS guidance for large runs
-

๐Ÿงฌ Organism-Agnostic#

-
  • Custom reference: --ref_annotation + --ref_genome replaces built-in H37Rv
  • Auto-detect mode: --skip_mycobrowser auto-enabled when genus โ‰  Mycobacterium
  • Dynamic reference comparison: report uses median distribution (not hardcoded H37Rv)
  • Auto-extending color palette: non-MTBC lineages get colors automatically
  • Generic labels: "Ratio to Median" instead of "Ratio to H37Rv"
  • Manifest updated: "Assisted Microbial Annotator Pipeline"
-

๐Ÿ“Š Output Quality#

-
  • Clean proteome: {sample}-AMAP.clean.faa without pseudogenes or internal stops
  • Annotation QC: per-sample metrics (genome size, GC%, coding density, hypothetical %, annotation rate)
  • GFF3 spec compliance: ##sequence-region pragmas, attribute escaping, sorted features
  • QC report section: interactive table + coding density / hypothetical % / genome size plots
-

๐Ÿงช Code Quality#

-
  • 58 unit tests: covering IDs, overlaps, reconciliation, validation, clean proteome, QC, merge
  • GitHub Actions CI: Python 3.9 + 3.11 matrix (currently disabled, ready to activate)
  • Type hints: core modules with Coord, Coord3, CoordDict aliases
  • Module docstrings: coordinate conventions, naming schemes documented
-

๐Ÿ–ฅ๏ธ Usability#

-
  • Structured startup banner: shows all configuration in a formatted box
  • --version flag: prints pipeline + Nextflow version
  • Progress logging: โ–ธ [BAKTA] Annotating G26689... for every major step
  • Actionable error messages: boxed errors with cause + fix + re-run command
  • Sample count at startup: confirms TSV was parsed correctly
-

๐Ÿ“ Documentation#

-
  • README: updated for v1.0.0 with all new parameters, outputs, and organism-agnostic usage
  • CHANGELOG: this file
  • Output tree: documents .clean.faa, .qc.tsv, pipeline_info/ new files
-

๐Ÿ› Bug Fixes (85 total)#

-
  • 20 critical (๐Ÿ”ด) + 65 improvements (๐ŸŸก)
  • See commit history on fix branch for full details
-

[0.1.0] - 2025#

-
  • Initial release
  • MTBC-focused annotation pipeline
  • Bakta + Miniprot + eggNOG integration
  • HTML report with COG, Mycobrowser, pangenome analysis
-

Edit this page on GitHub

- +

Correctness

#
+
  • Genes no longer vanish during overlap resolution: _insert_with_dedup queued entries for eviction while scanning for conflicts, and applied them even when it then abandoned the insert, so genes were deleted with nothing put in their place
  • Reference pseudogenes are recognised for any organism: the check was gated on the identifier starting with Rv, so on every other organism every reference pseudogene was extended to the next stop and lost its flag. The same gene came out differently depending on what it was called
  • The extracted reference proteome carries its stop codon: extract_ref_proteins.py wrote coordinates spanning the stop but a protein without it, so miniprot's alignment ended three bases short of every Bakta CDS and no gene could ever reach caseย 2. On E.ย coli O157:H7 this moved 3325 genes out of conflict resolution
  • Mycobrowser join key: built with rvs.split('_')[0], which returned the whole string for a gene with several references (Rv0094c, Rv3467) and truncated any name containing an underscore (PE_PGRS1 became PE). 44 more genes get a functional category on a real MTBC sample, none lose one
  • Annotation table and FASTA files agree: check_cds.py wrote the FASTA files from dictionaries filled before its final deduplication, and deduplicate_rvs.py matched FASTA headers on the gene id alone, so removing one copy of a multi-copy gene deleted all of them
  • Untranslatable genes are dropped from the table, not left in it with no sequence anywhere
+

Crashes

#
+
  • --skip_mycobrowser and organism-agnostic mode: the placeholder frame was built with a duplicate Locus column, so pd.merge refused to run and GET_OUTPUTS failed on every run without Mycobrowser
  • miniprot PAF parsing: pandas inferred the column count from the first row, and miniprot varies its trailing tags per alignment, so any file whose first row was narrower aborted the parse
  • Biopython 1.85 and newer: SeqFeature.strand was removed; to_panaroo.py and extract_ref_proteins.py now read it off the location
  • -profile test: pointed at a sample sheet naming paths that are not in this repository. It now annotates the reference genome bundled in the container, so it needs no data and nothing downloaded
+

Silent data loss

#
+
  • First Diamond hit: read_and_create left header inference on while stripping # comments, so the first alignment became the column names and disappeared from the duplication analysis
  • Multi-part gene names in rename_fastas.py: headers were cut at the third underscore, collapsing every PE/PPE gene of a sample onto one identifier
  • eggNOG annotation rate: counted with notna(), and emapper writes - for fields it could not fill, so the QC metric read close to 100% whatever the run had done
+

Performance

#
+
  • reconcile_coordinates is linear rather than quadratic: it scanned every protein and every reference for each gene, calling a regex on each candidate. Indexed by gene name up front, a 4000-gene genome goes from 18.3ย s to 0.011ย s with identical output over 200 randomised genomes
  • deduplicate: list.count() per element and a full scan per duplicated gene, replaced with a Counter and a single grouping pass
+

Report

#
+
  • Layout: the stylesheet was losing to the one rmarkdown inlines afterwards, so the article sat underneath the table of contents and the left logo on top of the title
  • Self-contained without a network at render time: jQuery came from a CDN and the second logo from GitHub. pandoc inlines those when the render machine has a network and leaves them dangling when it does not, so a report rendered on an isolated node arrived with a broken image
  • Half the size: 8.2ย MB to 4.4ย MB, by shipping only the plotly traces actually used and quantising the figures
  • The version in the report is the version that produced it: it was written by hand in three places in the Rmd, so any release that forgot to update them published a report naming the wrong version. It now comes from the manifest in nextflow.config
+

Documentation and infrastructure

#
+
  • Documentation site built with MkDocs Material, published to GitHub Pages, including a tutorial with the numbers from real runs and a rendered example report
  • Docker image workflow: builds, smoke tests and publishes on a v* tag or on demand
  • Tests workflow enabled: it had been sitting disabled and would have failed, since its dependency list was missing plotly and its syntax check had drifted five files behind
  • requirements-test.txt: one dependency list for CI, the docs and a local checkout. environment.yml was missing the Python plotly, so a conda environment built from it could not start classify_cases.py
  • 92 tests, up from 58
+

1.0.0, 1 April 2026

#
+

Correctness

#
+
  • Strand-aware gene dictionary: combined_dict keys changed from (Start, End) to (Start, End, Strand): genes on opposite strands at the same coordinates no longer collide silently
  • Consistent 1-based lengths: all length calculations now use End - Start + 1 (15 sites fixed)
  • Robust ID cleaning: regex _\d+-\d+$ replaces brittle 4-part underscore rule, so PE_PGRS genes are handled correctly
  • Exact gene name matching: prevents false matches (Rv3508 โ‰  Rv3508c, Rv1 โ‰  Rv10)
  • IntervalTree half-open intervals: all 9 operations corrected from [Start:End] to [Start:End+1]
  • Cross-strand gene preservation: merge operations now include Strand to prevent loss
  • Paralog preservation: multi-copy PE/PPE genes no longer collapsed during groupby
  • Pseudogene propagation: Is_Pseudogene column now reaches master table and report
+

Robustness

#
+
  • Pre-flight validation: new VALIDATE_INPUTS process checks FASTA format, N-content, GC%, genome size, duplicate headers and database paths, before any computation
  • --validate_only mode: check all inputs without executing the pipeline
  • eggNOG DB integrity: bacteria.dmnd <100ย MB detected as corrupted โ†’ automatic rebuild
  • Bakta DB pre-check: actionable error message with download command
  • OOM retry: memory scales with task.attempt (e.g., 50ย GB โ†’ 100ย GB โ†’ 150ย GB)
  • 0-protein handling: eggNOG batch handles empty inputs gracefully
  • Completion handler: prints duration, succeeded/failed/ignored counts
+

Reproducibility

#
+
  • nextflow.manifest: versionย 1.0.0, Nextflow โ‰ฅ23.04.0, DOI, homepage
  • SOFTWARE_VERSIONS: YAML + TSV with exact versions of 12 tools
  • DUMP_PARAMS: complete parameter dump per run
  • Diamond sorted output: deterministic results regardless of thread count
  • environment.yml: Conda environment with pinned versions
  • Louvain seed: fixed to 42 for reproducible community detection
+

Performance and HPC

#
+
  • Dynamic resource scaling: memory grows with each retry attempt
  • Node-local scratch: --scratch true (default on SLURM/SGE/PBS/LSF)
  • Diamond block size tuning: auto-calculated from available RAM
  • Bakta local temp: --tmp-dir on node-local disk (avoids NFS metadata storms)
  • eggNOG batch mode: single run for all samples (10ร— faster, 50ร— less filesystem I/O)
  • Thread utilization: Bakta 12, Diamond 6-24, miniprot 2ร—3, pangene 2ร—N
  • SLURM/SGE/PBS/LSF profiles: with time limits, queue sizes, rate limiting
  • storeDir for eggNOG DB: persists across runs, survives nextflow clean
+

Scalability

#
+
  • find -print0 | xargs -0: replaces echo|tr|grep in REPORT_INPUTS (handles 500+ files)
  • publishDir mode: 'link': hardlinks instead of copies for per-sample outputs
  • cache 'lenient': tolerates NFS timestamp drift between cluster nodes
  • awk close(): prevents file handle exhaustion on 500+ sample eggNOG splits
  • JVM tuning docs: NXF_OPTS guidance for large runs
+

Organism-agnostic

#
+
  • Custom reference: --ref_annotation + --ref_genome replaces built-in H37Rv
  • Auto-detect mode: --skip_mycobrowser auto-enabled when genus โ‰  Mycobacterium
  • Dynamic reference comparison: report uses median distribution (not hardcoded H37Rv)
  • Auto-extending color palette: non-MTBC lineages get colors automatically
  • Generic labels: "Ratio to Median" instead of "Ratio to H37Rv"
  • Manifest updated: "Assisted Microbial Annotator Pipeline"
+

Output quality

#
+
  • Clean proteome: {sample}-AMAP.clean.faa without pseudogenes or internal stops
  • Annotation QC: per-sample metrics (genome size, GC%, coding density, hypothetical %, annotation rate)
  • GFF3 spec compliance: ##sequence-region pragmas, attribute escaping, sorted features
  • QC report section: interactive table + coding density / hypothetical % / genome size plots
+

Code quality

#
+
  • 58 unit tests: covering IDs, overlaps, reconciliation, validation, clean proteome, QC, merge
  • GitHub Actions CI: Python 3.9 + 3.11 matrix (currently disabled, ready to activate)
  • Type hints: core modules with Coord, Coord3, CoordDict aliases
  • Module docstrings: coordinate conventions, naming schemes documented
+

Usability

#
+
  • Structured startup banner: shows all configuration in a formatted box
  • --version flag: prints pipeline + Nextflow version
  • Progress logging: โ–ธ [BAKTA] Annotating G26689... for every major step
  • Actionable error messages: boxed errors with cause + fix + re-run command
  • Sample count at startup: confirms TSV was parsed correctly
+

Documentation

#
+
  • README: updated for v1.0.0 with all new parameters, outputs, and organism-agnostic usage
  • CHANGELOG: this file
  • Output tree: documents .clean.faa, .qc.tsv, pipeline_info/ new files
+

Bug fixes (85 total)

#
+
  • 20 critical fixes and 65 improvements
+

0.1.0, 2025

#
+
  • Initial release
  • MTBC-focused annotation pipeline
  • Bakta + miniprot + eggNOG integration
  • HTML report with COG, Mycobrowser, pangenome analysis
+

Back to the top

+

Edit this page on GitHub

+
- - diff --git a/docs/about/citation.html b/docs/about/citation.html index 6f0aa83..6391e17 100644 --- a/docs/about/citation.html +++ b/docs/about/citation.html @@ -4,124 +4,228 @@ Citation | AMAP docs - + + - + - + - + - +
- AMAP + AMAP
- + + +
-

Docs / About

-

Citing AMAP

-

Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software]. https://doi.org/10.5281/zenodo.8024446

-
@software{Ruiz-Rodriguez_AMAP_2026,
+
Docs / About
Describes AMAP on main, ahead of 1.0.1: what changed

+
Citation

+
How to cite AMAP and the tools it runs.

+
1 min read

+
On this page

+
+
If AMAP is part of your work, cite it in whichever form your reference manager takes. All three say the same thing; the DOI is the concept DOI, which always resolves to the newest version.

+
Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software]. https://doi.org/10.5281/zenodo.8024446
+

The CITATION.cff tab is the repository's own file, which GitHub reads to offer a formatted citation from the sidebar.

+

Authors

#

Paula Ruiz-Rodriguez and Mireia Coscolla

-

I2SysBio, University of Valencia-CSIC, FISABIO Joint Research Unit Infection and Public Health, Valencia, Spain.

-

Cite the tools too#

+

Institute for Integrative Systems Biology (I2SysBio, University of Valencia-CSIC), FISABIO Joint Research Unit "Infection and Public Health", Paterna, Spain.

+

Cite the tools too

#

AMAP orchestrates other people's work. If a result depends on one of these, cite it as well.

-
ToolWhat it does here
NextflowRuns the pipeline
BaktaBaseline annotation
MiniprotReference proteome alignment
eggNOG-mapperFunctional annotation
DiamondDuplication searches
pangenePangenome graph
pathotyprMTBC lineage typing
-

The exact version of each one used by a given run is recorded in results/pipeline_info/software_versions.yml.

-

Reference data#

+
ToolWhat it does herePaper
NextflowRuns the pipelineDi Tommaso et al., Nature Biotechnology 35, 316-319 (2017). 10.1038/nbt.3820
BaktaBaseline annotationSchwengers et al., Microbial Genomics 7, 000685 (2021). 10.1099/mgen.0.000685
miniprotReference proteome alignmentLi, Bioinformatics 39, btad014 (2023). 10.1093/bioinformatics/btad014
eggNOG-mapperFunctional annotationCantalapiedra et al., Molecular Biology and Evolution 38, 5825-5829 (2021). 10.1093/molbev/msab293
DiamondDuplication searches, and eggNOG'sBuchfink et al., Nature Methods 18, 366-368 (2021). 10.1038/s41592-021-01101-x
pangenePangenome graphLi et al., Bioinformatics 40, btae456 (2024). 10.1093/bioinformatics/btae456
pathotyprLineage typing of the Mycobacterium tuberculosis complex (MTBC)Its repository, and its lineage markers below
+

All of them in one block, for a reference manager:

+
@article{DiTommaso_Nextflow_2017,
+  author = {Di Tommaso, Paolo and Chatzou, Maria and Floden, Evan W. and
+    Barja, Pablo Prieto and Palumbo, Emilio and Notredame, Cedric},
+  title = {Nextflow enables reproducible computational workflows},
+  journal = {Nature Biotechnology},
+  volume = {35},
+  number = {4},
+  pages = {316--319},
+  year = {2017},
+  doi = {10.1038/nbt.3820}}
+
+@article{Schwengers_Bakta_2021,
+  author = {Schwengers, Oliver and Jelonek, Lukas and Dieckmann, Marius Alfred and
+    Beyvers, Sebastian and Blom, Jochen and Goesmann, Alexander},
+  title = {Bakta: rapid and standardized annotation of bacterial genomes via alignment-free
+    sequence identification},
+  journal = {Microbial Genomics},
+  volume = {7},
+  number = {11},
+  pages = {000685},
+  year = {2021},
+  doi = {10.1099/mgen.0.000685}}
+
+@article{Li_miniprot_2023,
+  author = {Li, Heng},
+  title = {Protein-to-genome alignment with miniprot},
+  journal = {Bioinformatics},
+  volume = {39},
+  number = {1},
+  pages = {btad014},
+  year = {2023},
+  doi = {10.1093/bioinformatics/btad014}}
+
+@article{Cantalapiedra_eggNOGmapper_2021,
+  author = {Cantalapiedra, Carlos P. and Hern{\'a}ndez-Plaza, Ana and Letunic, Ivica and
+    Bork, Peer and Huerta-Cepas, Jaime},
+  title = {{eggNOG-mapper} v2: Functional Annotation, Orthology Assignments, and Domain
+    Prediction at the Metagenomic Scale},
+  journal = {Molecular Biology and Evolution},
+  volume = {38},
+  number = {12},
+  pages = {5825--5829},
+  year = {2021},
+  doi = {10.1093/molbev/msab293}}
+
+@article{Buchfink_DIAMOND_2021,
+  author = {Buchfink, Benjamin and Reuter, Klaus and Drost, Hajk-Georg},
+  title = {Sensitive protein alignments at tree-of-life scale using {DIAMOND}},
+  journal = {Nature Methods},
+  volume = {18},
+  number = {4},
+  pages = {366--368},
+  year = {2021},
+  doi = {10.1038/s41592-021-01101-x}}
+
+@article{Li_pangene_2024,
+  author = {Li, Heng and Marin, Maximillian and Farhat, Maha R.},
+  title = {Exploring gene content with pangene graphs},
+  journal = {Bioinformatics},
+  volume = {40},
+  number = {7},
+  pages = {btae456},
+  year = {2024},
+  doi = {10.1093/bioinformatics/btae456}}
+

The exact version of each one used by a given run is recorded in results/pipeline_info/software_versions.yml.

+

Reference data

#

The MTBC reference files bundled in the container come from public repositories. The pathotypr lineage markers are archived on Zenodo under 10.5281/zenodo.19210044, and Mycobrowser functional categories come from mycobrowser.epfl.ch.

-

Licence#

+

Licence

#

AMAP is released under the GPL-3.0 licence. See LICENSE.

-

Edit this page on GitHub

- +

Edit this page on GitHub

+
- - diff --git a/docs/about/citation.md b/docs/about/citation.md index ea37f52..f07e699 100644 --- a/docs/about/citation.md +++ b/docs/about/citation.md @@ -1,44 +1,154 @@ -# Citing AMAP +# Citation -> Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) -> [Computer software]. +If AMAP is part of your work, cite it in whichever form your reference manager +takes. All three say the same thing; the DOI is the concept DOI, which always +resolves to the newest version. + +**APA** + +```apa +Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software]. https://doi.org/10.5281/zenodo.8024446 +``` + +**BibTeX** ```bibtex @software{Ruiz-Rodriguez_AMAP_2026, author = {Ruiz-Rodriguez, Paula and Coscolla, Mireia}, doi = {10.5281/zenodo.8024446}, license = {GPL-3.0}, - month = apr, + month = jul, title = {{AMAP}}, url = {https://github.com/PathoGenOmics-Lab/AMAP}, version = {1.0.1}, year = {2026}} ``` -The repository also carries a `CITATION.cff`, which GitHub reads to offer a -formatted citation from the sidebar. +**CITATION.cff** + +```yaml +cff-version: 1.2.0 +title: AMAP +message: 'If you use this software, please cite it as below.' +type: software +authors: + - family-names: Ruiz-Rodriguez + given-names: Paula + orcid: 'https://orcid.org/0000-0003-0727-5974' + affiliation: >- + Institute for Integrative Systems Biology (I2SysBio, + University of Valencia - CSIC), FISABIO Joint Research + Unit โ€œInfection and Public Healthโ€, Paterna, Spain + email: paula.ruiz.rodriguez@csic.es + - family-names: Coscolla + given-names: Mireia + orcid: 'https://orcid.org/0000-0003-0752-0538' + affiliation: >- + Institute for Integrative Systems Biology (I2SysBio, + University of Valencia - CSIC), FISABIO Joint Research + Unit โ€œInfection and Public Healthโ€, Paterna, Spain + email: mireia.coscolla@csic.es +url: 'https://github.com/PathoGenOmics-Lab/AMAP' +license: GPL-3.0 +version: 1.0.1 +date-released: '2026-07-25' +doi: 10.5281/zenodo.8024446 +``` + +The CITATION.cff tab is the repository's own file, which GitHub reads to offer +a formatted citation from the sidebar. ## Authors **Paula Ruiz-Rodriguez** and **Mireia Coscolla** -I2SysBio, University of Valencia-CSIC, FISABIO Joint Research Unit -Infection and Public Health, Valencia, Spain. +Institute for Integrative Systems Biology (I2SysBio, University of +Valencia-CSIC), FISABIO Joint Research Unit "Infection and Public Health", +Paterna, Spain. ## Cite the tools too AMAP orchestrates other people's work. If a result depends on one of these, cite it as well. -| Tool | What it does here | -| :--- | :--- | -| [Nextflow](https://www.nextflow.io/) | Runs the pipeline | -| [Bakta](https://github.com/oschwengers/bakta) | Baseline annotation | -| [Miniprot](https://github.com/lh3/miniprot) | Reference proteome alignment | -| [eggNOG-mapper](https://github.com/eggnogdb/eggnog-mapper) | Functional annotation | -| [Diamond](https://github.com/bbuchfink/diamond) | Duplication searches | -| [pangene](https://github.com/lh3/pangene) | Pangenome graph | -| [pathotypr](https://github.com/PathoGenOmics-Lab/pathotypr) | MTBC lineage typing | +| Tool | What it does here | Paper | +| :--- | :--- | :--- | +| [Nextflow](https://www.nextflow.io/) | Runs the pipeline | Di Tommaso et al., *Nature Biotechnology* 35, 316-319 (2017). [10.1038/nbt.3820](https://doi.org/10.1038/nbt.3820) | +| [Bakta](https://github.com/oschwengers/bakta) | Baseline annotation | Schwengers et al., *Microbial Genomics* 7, 000685 (2021). [10.1099/mgen.0.000685](https://doi.org/10.1099/mgen.0.000685) | +| [miniprot](https://github.com/lh3/miniprot) | Reference proteome alignment | Li, *Bioinformatics* 39, btad014 (2023). [10.1093/bioinformatics/btad014](https://doi.org/10.1093/bioinformatics/btad014) | +| [eggNOG-mapper](https://github.com/eggnogdb/eggnog-mapper) | Functional annotation | Cantalapiedra et al., *Molecular Biology and Evolution* 38, 5825-5829 (2021). [10.1093/molbev/msab293](https://doi.org/10.1093/molbev/msab293) | +| [Diamond](https://github.com/bbuchfink/diamond) | Duplication searches, and eggNOG's | Buchfink et al., *Nature Methods* 18, 366-368 (2021). [10.1038/s41592-021-01101-x](https://doi.org/10.1038/s41592-021-01101-x) | +| [pangene](https://github.com/lh3/pangene) | Pangenome graph | Li et al., *Bioinformatics* 40, btae456 (2024). [10.1093/bioinformatics/btae456](https://doi.org/10.1093/bioinformatics/btae456) | +| [pathotypr](https://github.com/PathoGenOmics-Lab/pathotypr) | Lineage typing of the *Mycobacterium tuberculosis* complex (MTBC) | Its repository, and its lineage markers below | + +All of them in one block, for a reference manager: + +```bibtex +@article{DiTommaso_Nextflow_2017, + author = {Di Tommaso, Paolo and Chatzou, Maria and Floden, Evan W. and + Barja, Pablo Prieto and Palumbo, Emilio and Notredame, Cedric}, + title = {Nextflow enables reproducible computational workflows}, + journal = {Nature Biotechnology}, + volume = {35}, + number = {4}, + pages = {316--319}, + year = {2017}, + doi = {10.1038/nbt.3820}} + +@article{Schwengers_Bakta_2021, + author = {Schwengers, Oliver and Jelonek, Lukas and Dieckmann, Marius Alfred and + Beyvers, Sebastian and Blom, Jochen and Goesmann, Alexander}, + title = {Bakta: rapid and standardized annotation of bacterial genomes via alignment-free + sequence identification}, + journal = {Microbial Genomics}, + volume = {7}, + number = {11}, + pages = {000685}, + year = {2021}, + doi = {10.1099/mgen.0.000685}} + +@article{Li_miniprot_2023, + author = {Li, Heng}, + title = {Protein-to-genome alignment with miniprot}, + journal = {Bioinformatics}, + volume = {39}, + number = {1}, + pages = {btad014}, + year = {2023}, + doi = {10.1093/bioinformatics/btad014}} + +@article{Cantalapiedra_eggNOGmapper_2021, + author = {Cantalapiedra, Carlos P. and Hern{\'a}ndez-Plaza, Ana and Letunic, Ivica and + Bork, Peer and Huerta-Cepas, Jaime}, + title = {{eggNOG-mapper} v2: Functional Annotation, Orthology Assignments, and Domain + Prediction at the Metagenomic Scale}, + journal = {Molecular Biology and Evolution}, + volume = {38}, + number = {12}, + pages = {5825--5829}, + year = {2021}, + doi = {10.1093/molbev/msab293}} + +@article{Buchfink_DIAMOND_2021, + author = {Buchfink, Benjamin and Reuter, Klaus and Drost, Hajk-Georg}, + title = {Sensitive protein alignments at tree-of-life scale using {DIAMOND}}, + journal = {Nature Methods}, + volume = {18}, + number = {4}, + pages = {366--368}, + year = {2021}, + doi = {10.1038/s41592-021-01101-x}} + +@article{Li_pangene_2024, + author = {Li, Heng and Marin, Maximillian and Farhat, Maha R.}, + title = {Exploring gene content with pangene graphs}, + journal = {Bioinformatics}, + volume = {40}, + number = {7}, + pages = {btae456}, + year = {2024}, + doi = {10.1093/bioinformatics/btae456}} +``` The exact version of each one used by a given run is recorded in `results/pipeline_info/software_versions.yml`. diff --git a/docs/about/contributing.html b/docs/about/contributing.html index 7bbf99e..ec5b810 100644 --- a/docs/about/contributing.html +++ b/docs/about/contributing.html @@ -4,154 +4,169 @@ Contributing | AMAP docs - + + - + - + - + - +
- AMAP + AMAP
- + + +
-

Docs / About

+

Docs / About

Describes AMAP on main, ahead of 1.0.1: what changed

Contributing

+

Working on the code, the tests and these pages.

+

5 min read

+
On this page
+

Bug reports, fixes and suggestions are all welcome. The full guidelines live in CONTRIBUTING.md, and the project follows a code of conduct.

-

Reporting a bug#

-

Open an issue at github.com/PathoGenOmics-Lab/AMAP/issues with the command you ran, the failing task's .command.err, and both software_versions.yml and params.yml from pipeline_info/. Those last two pin the exact versions and parameters, which is usually what the answer turns on. Troubleshooting covers the common cases first.

-

Working on the code#

-
git clone https://github.com/PathoGenOmics-Lab/AMAP
-cd AMAP
-pip install -r requirements-test.txt
-python -m pytest tests/ -v
-

requirements-test.txt covers everything bin/ imports, not only what the tests import directly, and CI installs from the same file. A shorter list appears to work until a test touches classify_cases.py, which pulls in plotly through classify/plotting.py.

-

The suite covers ID handling, overlap arithmetic, case classification, the rules that resolve and recover genes, and the parsers, and it runs in about a second. Anything touching bin/classify/ should come with a test, since the correction logic is where a silent change does the most damage.

-

Annotating H37Rv with itself#

+

Reporting aย bug

#
+

Open an issue at github.com/PathoGenOmics-Lab/AMAP/issues with the command you ran, the failing task's .command.err, and both software_versions.yml and params.yml from pipeline_info/. Those last two pin the exact versions and parameters, which is usually what the answer turns on. Troubleshooting covers the common cases first.

+

Working on the code

#
+
Terminal
$ git clone https://github.com/PathoGenOmics-Lab/AMAP
+$ cd AMAP
+$ pip install -r requirements-test.txt
+$ python -m pytest tests/ -v
+

requirements-test.txt covers everything bin/ imports, not only what the tests import directly, and CI installs from the same file. A shorter list appears to work until a test touches classify_cases.py, which pulls in plotly through classify/plotting.py.

+

The suite covers ID handling, overlap arithmetic, case classification, the rules that resolve and recover genes, and the parsers, and it runs in about three minutes (some tests skip themselves when a tool such as R or Diamond is missing). Anything touching bin/classify/ should come with a test, since the correction logic is where a silent change does the most damage.

+

Annotating H37Rv with itself

#

A change to the correction algorithm should also pass the self-test, which has exactly one right answer: annotating H37Rv against its own reference must give back every CDS of the AL123456.3 GenBank record, at exactly its coordinates and strand, under its own name.

-
docker run --rm --platform linux/amd64 -v "$PWD:/amap" -w /amap \
-    paururo/amap:latest bash tests/selftest/h37rv_selftest.sh
-
GenBank CDS: 4031   AMAP genes: 4031
+
Terminal
$ docker run --rm --platform linux/amd64 -v "$PWD:/amap" -w /amap \
+    paururo/amap:latest bash tests/selftest/h37rv_selftest.sh

+
Output
GenBank CDS: 4031   AMAP genes: 4031
   exact and named right: 4031
   not at exactly their coordinates: 0
   named wrong: 0
   AMAP genes that are no GenBank CDS: 0

-
The first run spends a couple of minutes on Bakta and miniprot and keeps their output under selftest_h37rv/. After that it only runs classify_cases.py, check_cds.py and deduplicate_rvs.py, in seconds, and exits non-zero with the offending genes listed if anything is missing, misplaced or misnamed.

-
H37Rv is one genome, and the one the defaults were tuned on. The accuracy benchmark in tests/benchmark/ scores six more whose right answer is known: another H37Rv assembly, CDC1551 complete and cut into 40 contigs, M. bovis, and E. coli K-12 and Sakai against K-12 as a custom reference. Give a run a name, and the next one lists every gene that changed:

-
docker run --rm --platform linux/amd64 -v "$PWD:/amap" -w /amap \
-    paururo/amap:latest bash tests/benchmark/benchmark.sh benchmark_work before
-# change the algorithm, then
-docker run --rm --platform linux/amd64 -v "$PWD:/amap" -w /amap \
-    paururo/amap:latest bash tests/benchmark/benchmark.sh benchmark_work after before

-
tests/benchmark/README.md explains how the truth is built and what to expect.

-
The quickest way to run the whole pipeline is the test profile:

-
nextflow run . -profile docker,test

-
Pipeline code and -resume#

-
Nextflow hashes a task's inputs and script, not the files the script calls by path, so python3 ${projectDir}/bin/check_cds.py alone would let -resume hand back what the old check_cds.py produced. Every process that runs AMAP code therefore lists those files in ext code, and its script names task.ext.code in a comment, which puts them in the cache key (Nextflow 24.04 and later):

-
ext code: { files("${projectDir}/bin/{check_cds.py,deduplicate_rvs.py,classify/*.py}") }

-
A script that imports classify needs classify/*.py in the list. The reverse also bites: Nextflow adds to a task's cache key any bin/ entry its script names as a bare word, which is why PATHOTYPR quotes its 'classify' subcommand. tests/test_cache_keys.py fails when a process runs a file it does not list, lists one that does not exist, or names a bin/ entry it does not declare.

-
Working on the container#

-
docker build --platform linux/amd64 -f docker/Dockerfile -t amap:dev .
-docker/smoke_test.sh amap:dev

+
The first run spends a couple of minutes on Bakta and miniprot and keeps their output under selftest_h37rv/. After that it only runs classify_cases.py, check_cds.py and deduplicate_rvs.py, in seconds, and exits non-zero with the offending genes listed if anything is missing, misplaced or misnamed.

+
H37Rv is one genome, and the one the defaults were tuned on. The accuracy benchmark in tests/benchmark/ scores six more whose right answer is known: another H37Rv assembly, CDC1551 complete and cut into 40 contigs, M.ย bovis, and E.ย coli K-12 and Sakai against K-12 as a custom reference. Give a run a name, and the next one lists every gene that changed:

+
Terminal
$ docker run --rm --platform linux/amd64 -v "$PWD:/amap" -w /amap \
+    paururo/amap:latest bash tests/benchmark/benchmark.sh benchmark_work before
+# change the algorithm, then
+$ docker run --rm --platform linux/amd64 -v "$PWD:/amap" -w /amap \
+    paururo/amap:latest bash tests/benchmark/benchmark.sh benchmark_work after before

+
tests/benchmark/README.md explains how the truth is built and what to expect.

+
The quickest check of the whole chain up to the correction is the test profile, without eggNOG, whose 45ย GB database the test profile's 8ย GB cap cannot hold:

+
Terminal
$ nextflow run . -profile docker,test --skip_functional_annotation --outdir amap-test

+
Pipeline code and -resume
#

+
Nextflow hashes a task's inputs and script, not the files the script calls by path, so python3 ${projectDir}/bin/check_cds.py alone would let -resume hand back what the old check_cds.py produced. Every process that runs AMAP code therefore lists those files in ext code, and its script names task.ext.code in a comment, which puts them in the cache key (Nextflowย 24.04 and later):

+
ext code: { files("${projectDir}/bin/{check_cds.py,deduplicate_rvs.py,classify/*.py}") }

+
A script that imports classify needs classify/*.py in the list. The reverse also bites: Nextflow adds to a task's cache key any bin/ entry its script names as a bare word, which is why PATHOTYPR quotes its 'classify' subcommand. tests/test_cache_keys.py fails when a process runs a file it does not list, lists one that does not exist, or names a bin/ entry it does not declare.

+
Working on the container
#

+
Terminal
$ docker build --platform linux/amd64 -f docker/Dockerfile -t amap:dev .
+$ docker/smoke_test.sh amap:dev

 
The smoke test checks the tools, the Python and R libraries, the bundled reference data, and runs the test suite with the image interpreter. Worth doing before proposing a change to the Dockerfile, since a library update can break the pipeline scripts without the image looking any different.

-
Working on these docs#

-
The website is plain HTML in docs/, with nothing to build on the server: GitHub Pages serves the folder as it is (docs/.nojekyll keeps it from running Jekyll over the markdown), and it opens from a clone too. The long pages are the markdown files, which GitHub also renders, and tools/build_docs.py writes each one's .html next to it, with the site's header, page list, contents and search.

-
python3 tools/build_docs.py          # after editing any .md under docs/
-python3 tools/check_site.py          # what CI runs
-python3 -m http.server -d docs 8000  # then open http://localhost:8000

-
The front page (docs/index.html) and the illustrated walk through the correction (docs/how-it-works.html) are written by hand. Their drawings come from docs/assets/figures.js and how.js, and every gene in them from the windows of real runs in docs/assets/figure-data.js. A line <!-- figure: NAME --> in a markdown page puts one of those drawings there, and GitHub shows nothing in its place.

-
Some drawings are made from the code itself when the page is built, so they cannot drift from it: the sample sheet and its error messages run bin/validate_inputs.py, the validation ranges read its limits, the container reads docker/Dockerfile, and the cluster and failure drawings read nextflow.config. After changing any of those, run build_docs.py again.

-
check_site.py fails when a page no longer matches its markdown, a link or an anchor is broken, a page uses an option the pipeline does not have, a threshold on a page differs from the code, the metro map's stations are out of the order main.nf runs them in, or a drawing shows something bin/ would not do: it runs every window of figure-data.js through the classification, check_cds.py and deduplicate_rvs.py. When a change to the algorithm changes a drawn window, rerun the benchmark, write the windows again, and read the words of the drawing against what changed:

-
python3 tools/figure_data.py benchmark_work final --reference /reference

-
The duplications page draws a real family and a pair of communities from docs/assets/dup-data.js, which check_site.py puts back through get_duplications.py and network_genomes.py. tools/dup_run.sh runs the two duplication steps in the container on corrected proteomes, and tools/dup_data.py writes the data from what they found (both explain their arguments). The front page's benchmark scores are checked against tests/benchmark/README.md.

-
The pictures, the example report's sections, the card a shared link shows and the README's light and dark drawings, are screenshots of the site taken by headless Chrome; take them again after the report or a drawing changes:

-
node tools/pictures.mjs

-
The workflow in .github/workflows/docs.yml runs both checks on every pull request that touches the docs, bin/, modules/, main.nf, nextflow.config, docker/Dockerfile or the benchmark's expected results.

-
Style#

-
Keep to what is already there: short paragraphs, GitHub's alerts (> [!WARNING]) for the things that actually bite, and concrete numbers rather than "large" or "slow". Where a threshold appears in the docs it should match the constant in the code, so if you change one, change the other; check_site.py holds the ones it knows to the code.

-
Edit this page on GitHub

-
+
Working on these docs
#

+
The website is plain HTML in docs/, with nothing to build on the server: GitHub Pages serves the folder as it is (docs/.nojekyll keeps it from running Jekyll over the markdown), and it opens from a clone too. The long pages are the markdown files, which GitHub also renders, and tools/build_docs.py writes each one's .html next to it, with the site's header, page list, contents and search.

+
Terminal
$ python3 tools/build_docs.py          # after editing any .md under docs/
+$ python3 tools/check_site.py          # what CI runs
+$ python3 -m http.server -d docs 8000  # then open http://localhost:8000

+
The front page (docs/index.html) and the illustrated walk through the correction (docs/how-it-works.html) are written by hand. Their drawings come from docs/assets/figures.js and how.js, and every gene in them from the windows of real runs in docs/assets/figure-data.js. A line <!-- figure: NAME --> in a markdown page puts one of those drawings there, and GitHub shows nothing in its place.

+
Some drawings are made from the code itself when the page is built, so they cannot drift from it: the sample sheet and its error messages run bin/validate_inputs.py, the validation ranges read its limits, the container reads docker/Dockerfile, and the cluster and failure drawings read nextflow.config. After changing any of those, run build_docs.py again.

+
check_site.py fails when a page no longer matches its markdown, a link or an anchor is broken, a page uses an option the pipeline does not have, a threshold on a page differs from the code, the metro map's stations are out of the order main.nf runs them in, or a drawing shows something bin/ would not do: it runs every window of figure-data.js through the classification, check_cds.py and deduplicate_rvs.py. When a change to the algorithm changes a drawn window, rerun the benchmark, write the windows again, and read the words of the drawing against what changed:

+
Terminal
$ python3 tools/figure_data.py benchmark_work final --reference /reference

+
The duplications page draws a real family and a pair of communities from docs/assets/dup-data.js, which check_site.py puts back through get_duplications.py and network_genomes.py. tools/dup_run.sh runs the two duplication steps in the container on corrected proteomes, and tools/dup_data.py writes the data from what they found (both explain their arguments). The front page's benchmark scores are checked against tests/benchmark/README.md.

+
The pictures, the example report's sections, the card a shared link shows and the README's light and dark drawings are screenshots of the site taken by headless Chrome; take them again after the report or a drawing changes:

+
Terminal
$ node tools/pictures.mjs

+
The workflow in .github/workflows/docs.yml runs both checks on every pull request that touches the docs, bin/, modules/, main.nf, nextflow.config, docker/Dockerfile or the benchmark's expected results.

+
Style
#

+
Keep to what is already there: short paragraphs, GitHub's alerts (> [!WARNING]) for the things that actually bite, and concrete numbers rather than "large" or "slow". Where a threshold appears in the docs it should match the constant in the code, so if you change one, change the other; check_site.py holds the ones it knows to the code.

+
Edit this page on GitHub

+
- - diff --git a/docs/about/contributing.md b/docs/about/contributing.md index 317589a..6729593 100644 --- a/docs/about/contributing.md +++ b/docs/about/contributing.md @@ -30,7 +30,7 @@ appears to work until a test touches `classify_cases.py`, which pulls in plotly through `classify/plotting.py`. The suite covers ID handling, overlap arithmetic, case classification, the -rules that resolve and recover genes, and the parsers, and it runs in about a second. Anything +rules that resolve and recover genes, and the parsers, and it runs in about three minutes (some tests skip themselves when a tool such as R or Diamond is missing). Anything touching `bin/classify/` should come with a test, since the correction logic is where a silent change does the most damage. @@ -61,8 +61,8 @@ the offending genes listed if anything is missing, misplaced or misnamed. H37Rv is one genome, and the one the defaults were tuned on. The accuracy benchmark in `tests/benchmark/` scores six more whose right answer is known: -another H37Rv assembly, CDC1551 complete and cut into 40 contigs, M. bovis, and -E. coli K-12 and Sakai against K-12 as a custom reference. Give a run a name, +another H37Rv assembly, CDC1551 complete and cut into 40 contigs, *M. bovis*, and +*E. coli* K-12 and Sakai against K-12 as a custom reference. Give a run a name, and the next one lists every gene that changed: ```bash @@ -75,10 +75,12 @@ docker run --rm --platform linux/amd64 -v "$PWD:/amap" -w /amap \ `tests/benchmark/README.md` explains how the truth is built and what to expect. -The quickest way to run the whole pipeline is the test profile: +The quickest check of the whole chain up to the correction is the test +profile, without eggNOG, whose 45 GB database the test profile's 8 GB cap +cannot hold: ```bash -nextflow run . -profile docker,test +nextflow run . -profile docker,test --skip_functional_annotation --outdir amap-test ``` ### Pipeline code and `-resume` @@ -161,7 +163,7 @@ arguments). The front page's benchmark scores are checked against `tests/benchmark/README.md`. The pictures, the example report's sections, the card a shared link shows and -the README's light and dark drawings, are screenshots of the site taken by +the README's light and dark drawings are screenshots of the site taken by headless Chrome; take them again after the report or a drawing changes: ```bash diff --git a/docs/assets/amap-logo-light.png b/docs/assets/amap-logo-light.png new file mode 100644 index 0000000..464d9ba Binary files /dev/null and b/docs/assets/amap-logo-light.png differ diff --git a/docs/assets/favicon-light.png b/docs/assets/favicon-light.png new file mode 100644 index 0000000..a9df380 Binary files /dev/null and b/docs/assets/favicon-light.png differ diff --git a/docs/assets/figures.js b/docs/assets/figures.js index b2d6833..14f8ba1 100644 --- a/docs/assets/figures.js +++ b/docs/assets/figures.js @@ -7,7 +7,11 @@ (function () { "use strict"; - var reduceMotion = window.matchMedia && window.matchMedia("(prefers-reduced-motion: reduce)").matches; + function mq(query) { + return !!(window.matchMedia && window.matchMedia(query).matches); + } + + var reduceMotion = mq("(prefers-reduced-motion: reduce)"); function esc(text) { return String(text) @@ -17,10 +21,41 @@ .replace(/"/g, """); } + // The words that stay together, as tools/build_docs.py keeps them in the + // docs: a number and its unit, a word and the number it names (case 4), + // and the two ends of a range (67 to 90%). + function tie(text) { + return String(text) + .replace(/(\d) (?=(?:bp|kb|Mb|GB|MB|KB|ms|s|min|h|%|CPUs?|cores|residues|bytes)(?!\w))/g, "$1\u00a0") + .replace(/\b(case|Case|step|Step|Figure|version|Nextflow|Java) (?=\d)/g, "$1\u00a0") + .replace(/(\d) to (?=\d)/g, "$1\u00a0to\u00a0"); + } + function fmt(n) { return String(n).replace(/\B(?=(\d{3})+(?!\d))/g, ","); } + // A tooltip's HTML as one line a screen reader can say. + function plain(html) { + // A line break of the tip reads as a new sentence before a capital and + // as a comma before anything else. + return String(html) + .replace(/\s*\s*(<[^>]+>)*\s*(.)/gi, function (_m, _tags, next) { + return (/[A-Z]/.test(next) ? ". " : ", ") + next; + }) + .replace(/<[^>]+>/g, "") + .replace(/</g, "<") + .replace(/>/g, ">") + .replace(/"/g, '"') + .replace(/&/g, "&") + .replace(/(\d)-(\d)/g, "$1 to $2") + .replace(/\(\+\)/g, "plus strand") + .replace(/\(โˆ’\)/g, "minus strand") + .replace(/([.:;,])\s*\./g, "$1") + .replace(/\s+/g, " ") + .trim(); + } + /* โ”€โ”€ The track renderer โ”€โ”€ A drawing is rows of genes on one coordinate axis: @@ -28,31 +63,38 @@ ribbons: [{ a: [row, gene], b: [row, gene], kind, phase }], marks: [{ row, s, e, label }], edges: [{ row, at, side, label }] } A gene is { s, e, st, name, cls, tip, open, gap, dim, id }, 1-based and - inclusive like everything AMAP writes. */ + inclusive like everything AMAP writes. Drawing a spec leaves on it the + height it took (_h) and the kinds of mark it used (_used), for a legend + that names exactly those. */ var figureCount = 0; - function arrow(x1, x2, y, h, strand, open) { + // open: the ends a contig cut, drawn ragged. flat: the end an alignment + // stops short of, drawn square. + function arrow(x1, x2, y, h, strand, open, flat) { var tip = Math.min(8, Math.max(0, (x2 - x1) * 0.45)); var mid = y + h / 2; + var has = function (s, end) { + return !!s && s.indexOf(end) !== -1; + }; // A partial gene has no arrowhead at its cut end, and a ragged edge there. var zig = function (x, dir) { var d = 3 * dir; return "L" + (x + d) + "," + (y + h * 0.25) + "L" + x + "," + mid + "L" + (x + d) + "," + (y + h * 0.75); }; if (strand === "-") { - var rightOpen = open && open.indexOf("5'") !== -1; // 5' end is on the right - var leftOpen = open && open.indexOf("3'") !== -1; + var rightOpen = has(open, "5'"); // 5' end is on the right + var leftOpen = has(open, "3'"); var p = "M" + x2 + "," + y; - p += leftOpen ? "H" + x1 + zig(x1, 1) + "L" + x1 + "," + (y + h) : "H" + (x1 + tip) + "L" + x1 + "," + mid + "L" + (x1 + tip) + "," + (y + h); + p += leftOpen ? "H" + x1 + zig(x1, 1) + "L" + x1 + "," + (y + h) : has(flat, "3'") ? "H" + x1 + "V" + (y + h) : "H" + (x1 + tip) + "L" + x1 + "," + mid + "L" + (x1 + tip) + "," + (y + h); p += "H" + x2; p += rightOpen ? zig(x2, -1) + "Z" : "Z"; return p; } - var endOpen = open && open.indexOf("3'") !== -1; - var startOpen = open && open.indexOf("5'") !== -1; + var endOpen = has(open, "3'"); + var startOpen = has(open, "5'"); var q = "M" + x1 + "," + y; - q += endOpen ? "H" + x2 + zig(x2, -1) + "L" + x2 + "," + (y + h) : "H" + (x2 - tip) + "L" + x2 + "," + mid + "L" + (x2 - tip) + "," + (y + h); + q += endOpen ? "H" + x2 + zig(x2, -1) + "L" + x2 + "," + (y + h) : has(flat, "3'") ? "H" + x2 + "V" + (y + h) : "H" + (x2 - tip) + "L" + x2 + "," + mid + "L" + (x2 - tip) + "," + (y + h); q += "H" + x1; q += startOpen ? zig(x1, 1) + "Z" : "Z"; return q; @@ -67,17 +109,38 @@ return (o.from ? ' data-from="' + o.from + '"' : "") + (o.until ? ' data-until="' + o.until + '"' : ""); } + function r1(v) { + return Math.round(v * 10) / 10; + } + function draw(spec, width) { var id = "f" + ++figureCount; + var used = {}; // Drawn at the width it is shown at, so its text is always the page's size. - var W = Math.max(280, Math.round(width || spec.w || 640)); + var W = Math.round(width || spec.w || 640); var labelW = spec.labelW == null ? 104 : spec.labelW; + // On a phone the row names take only the room they need and the genes + // the rest, rather than the whole drawing shrinking under its words. + if (W < 280) { + if (labelW > 0) { + var need = 0; + spec.rows.forEach(function (row) { + if (row.overlay) return; + String(row.label).split("\n").forEach(function (line) { + need = Math.max(need, textWidth(line, 11)); + }); + }); + labelW = Math.min(labelW, Math.ceil(need) + 12); + } + W = Math.max(200, W); + } var x0 = labelW; var x1 = W - (spec.padRight == null ? 8 : spec.padRight); var geneH = spec.geneH || 16; var rowGap = spec.rowGap || 30; var top = spec.top == null ? 10 : spec.top; var span = spec.to - spec.from; + var live = !spec.inert; function X(p) { return x0 + ((p - spec.from) / span) * (x1 - x0); } @@ -119,17 +182,30 @@ return ys[r] + (lanes[r].of[k] || 0) * LANE; } var height = y - rowGap + (spec.axis ? 30 : 8); + spec._h = Math.round(height); var out = []; + // A drawing is one stop for the keyboard (a group of named genes); a small + // one inside a link is a picture, and only the link is a stop. out.push( - '' + '" ); out.push( - '' + - '' + "" + + '' + + '' + + '' + + '' + + '' + + '' + + "" ); + var masks = []; // Ribbons first, under the genes: straight where the two agree, slanted - // where their coordinates differ. + // where their coordinates differ. Their sides are drawn in full colour, + // solid where they agree and dashed where they do not, so the difference + // shows by shape as well as by hue. (spec.ribbons || []).forEach(function (r) { var ga = spec.rows[r.a[0]].genes[r.a[1]]; var gb = spec.rows[r.b[0]].genes[r.b[1]]; @@ -138,32 +214,49 @@ var clip = function (p) { return Math.max(spec.from, Math.min(spec.to, p)); }; - var pts = [X(clip(ga.s) - 0.5), ya, X(clip(ga.e) + 0.5), ya, X(clip(gb.e) + 0.5), yb, X(clip(gb.s) - 0.5), yb]; + var ax1 = r1(X(clip(ga.s) - 0.5)); + var ax2 = r1(X(clip(ga.e) + 0.5)); + var bx2 = r1(X(clip(gb.e) + 0.5)); + var bx1 = r1(X(clip(gb.s) - 0.5)); + used[r.kind] = 1; out.push( - '' + '" + + '' + + '' + + "" ); }); + // The frames of the marks, so a name set beside a gene keeps off them. + var frames = []; (spec.marks || []).forEach(function (m) { var mx1 = X(m.s - 0.5); var mx2 = X(m.e + 0.5); var my1 = ys[m.rows[0]] - 4; var my2 = ys[m.rows[1]] + geneH + (lanes[m.rows[1]].n - 1) * LANE + 4; - out.push('"); - out.push(''); + frames.push({ x1: mx1 - 1, x2: Math.max(mx1 + 2, mx2) + 1, y1: my1 - 1, y2: my2 + 1 }); + // A mark with a label says what it is; only one without needs the key. + if (m.kind === "ref") used["diff-ref"] = 1; + else if (!m.label) used.diff = 1; + out.push('"); + out.push( + '' + ); if (m.label) { var lx = m.labelAt === "left" ? mx1 - 4 : m.labelAt === "right" ? mx2 + 4 : (mx1 + mx2) / 2; var anchor = m.labelAt === "left" ? "end" : m.labelAt === "right" ? "start" : "middle"; - // A label that would run off either side is pinned to that side. + // A label that would run off either side is pinned to that side. Over + // the first row it may start in the empty top of the names' column, + // so it keeps its end at what it marks. var lw = textWidth(m.label, 10.5); var left = anchor === "end" ? lx - lw : anchor === "middle" ? lx - lw / 2 : lx; + var floor = m.rows[0] === 0 ? 0 : x0; if (left + lw > W - 2) { lx = W - 2; anchor = "end"; - } else if (left < x0) { - lx = x0; + } else if (left < floor) { + lx = floor; anchor = "start"; } out.push('' + esc(m.label) + ""); @@ -171,18 +264,44 @@ out.push(""); }); + // Every gene's box, so a name set beside a gene can keep off the others. + var boxes = []; + spec.rows.forEach(function (row, i) { + row.genes.forEach(function (g, k) { + boxes.push({ r: i, k: k, x1: X(Math.max(g.s, spec.from) - 0.5), x2: X(Math.min(g.e, spec.to) + 0.5), y1: gy0(i, k), y2: gy0(i, k) + geneH }); + }); + }); + function clear(x1, x2, y1, y2, r, k) { + if (x1 < x0 || x2 > W - 2 || y1 < 0) return false; + return boxes.every(function (b) { + return (b.r === r && b.k === k) || b.x2 <= x1 || b.x1 >= x2 || b.y2 <= y1 || b.y1 >= y2; + }) && frames.every(function (f) { + return f.x2 <= x1 || f.x1 >= x2 || f.y2 <= y1 || f.y1 >= y2; + }); + } + + var tabStop = !live; spec.rows.forEach(function (row, i) { var gy = ys[i]; // The label and the line stay while a row is swapped for a later // version of itself (an overlay), so they only take its first step. - out.push('"); + // Until then they show faintly, so the reader sees where it will come. + out.push('"); if (labelW > 0 && !row.overlay) { var lines = String(row.label).split("\n"); + var widest = 0; lines.forEach(function (line, k) { + widest = Math.max(widest, textWidth(line, 11)); out.push( '' + esc(line) + "" ); }); + // A dotted leader from the name to where its row starts, where + // there is room for one. + var lead1 = widest + 6; + if (x0 - 2 - lead1 >= 8) { + out.push(''); + } } if (!row.overlay) { out.push(''); @@ -192,7 +311,24 @@ (row.empty ? [row.empty] : []).forEach(function (note) { out.push('' + esc(note) + ""); }); - row.genes.forEach(function (g, k) { + // The keyboard starts at the leftmost gene of the first row. + var first = -1; + if (!tabStop && row.genes.length) { + first = 0; + row.genes.forEach(function (g, k) { + if (g.s < row.genes[first].s) first = k; + }); + tabStop = true; + } + // Wide genes first, so a narrow one is drawn over its neighbours and + // its target is never under theirs. + var order = row.genes.map(function (g, k) { + return k; + }).sort(function (a, b) { + return row.genes[b].e - row.genes[b].s - (row.genes[a].e - row.genes[a].s); + }); + order.forEach(function (k) { + var g = row.genes[k]; var gy = gy0(i, k); // A gene that runs past the window is cut at its edge, left open there. var cutL = g.s < spec.from; @@ -205,9 +341,43 @@ var cls = g.cls || row.cls; var klass = "g g-" + cls; var fill = cls === "pseudo" ? ' style="fill:url(#hatch-' + id + ')"' : ""; - var group = '"; - out.push(group); - out.push(''); + // A call with nothing aligned is drawn as a call until the calls are + // compared, then dashed, and grey if dropped. A figure that only + // shows the end (final) never shows the first. + var c1 = spec.phased && g.phaseCls && g.phaseCls.indexOf("c1-call") !== -1; + if (!(c1 && spec.final)) used[cls === "drop" && g.lk ? g.lk : cls] = 1; + if (g.short) used.short = 1; + if (g.partial) used.partial = 1; + if (c1) used[g.phaseCls.indexOf("ph-dim") !== -1 ? "drop" : "c1"] = 1; + var attrs = live + ? ' role="img" tabindex="' + (k === first ? 0 : -1) + '" aria-label="' + esc(plain(g.tip || g.name || "")) + '"' + (g.tip ? ' data-tip="' + esc(g.tip) + '"' : "") + : ""; + out.push('"); + if (live) { + // A target at least 14 px wide and 24 tall, whatever the gene's size. + var hw = Math.max(14, gx2 - gx1 + 4); + var hh = Math.max(24, geneH + 8); + out.push(''); + } + var maskAttr = ""; + if (g.short && gx2 - gx1 > 6) { + // An alignment short of its whole protein fades out on the side it + // lacks: its end (3'), or for a piece of a cut gene its start (5'). + // The 3' end is on the right of a plus-strand gene. + var mid = id + "-" + i + "-" + k; + var fadeW = Math.min(16, (gx2 - gx1) * 0.4); + var right = (g.short === "3'") === (g.st !== "-"); + var fx1 = right ? gx2 - fadeW : gx1 + fadeW; + var fx2 = right ? gx2 : gx1; + masks.push( + '' + + '' + + '' + + '' + ); + maskAttr = ' mask="url(#sm-' + mid + ')"'; + } + out.push(''); if (g.gap) { var gpx = X(g.gap); out.push(''); @@ -215,6 +385,12 @@ out.push('' + esc(g.gapLabel) + ""); } } + // A notch over each end AMAP moved away from Bakta's call. + (g.moved || []).forEach(function (at) { + var nx = X(at); + used.moved = 1; + out.push(''); + }); if (g.name) { var size = 11.5; var tw = textWidth(g.name, size); @@ -226,10 +402,25 @@ nx = gpx2 - gx1 > gx2 - gpx2 ? (gx1 + gpx2) / 2 : (gpx2 + gx2) / 2; } out.push( - '' + esc(g.name) + "" + '' + esc(g.name) + "" ); } else if (g.nameAbove) { - out.push('' + esc(g.name) + ""); + // Too narrow for its name, and the gene the figure is about: the + // name goes over it, kept inside the drawing, or beside it where + // over it would cross another gene. Where neither has room, the + // tip names it. + var aw = textWidth(g.name, 10.5); + var ax = Math.max(x0 + aw / 2, Math.min(W - 2 - aw / 2, (gx1 + gx2) / 2)); + var spots = [ + [ax, gy - 5, "middle", ax - aw / 2, ax + aw / 2, gy - 15, gy - 2], + [gx2 + 4, gy + geneH / 2 + 4, "start", gx2 + 2, gx2 + 6 + aw, gy + 1, gy + geneH - 1], + [gx1 - 4, gy + geneH / 2 + 4, "end", gx1 - 6 - aw, gx1 - 2, gy + 1, gy + geneH - 1] + ].filter(function (o) { + return clear(o[3], o[4], o[5], o[6], i, k); + }); + if (spots.length) { + out.push('' + esc(g.name) + ""); + } } } out.push(""); @@ -249,7 +440,7 @@ if (spec.axis) { var ay = y - rowGap + 10; - out.push(''); + out.push('"); } + if (masks.length) out.splice(2, 0, "" + masks.join("") + ""); out.push(""); + spec._used = used; return out.join(""); } @@ -286,56 +483,410 @@ return (n < 1.5 ? 1 : n < 3.5 ? 2 : n < 7.5 ? 5 : 10) * pow; } - // A tooltip for whatever in a drawing carries data-tip. - function attachTips(container) { - var tip = container.querySelector(".demo-tip"); + /* โ”€โ”€ What can be seen โ”€โ”€ + + A layer of the hero waits for its phase, and an element of a story for + its step; until then it is hidden, and takes neither the pointer nor the + keyboard. */ + + function isShown(el) { + if (el.closest(".step-off")) return false; + var svg = el.closest("svg"); + var phase = svg ? +svg.getAttribute("data-phase") : 0; + if (phase) { + if (phase < 2 && el.closest(".ph-ref")) return false; + if (phase < 3 && el.closest(".ph-case")) return false; + if (phase < 4 && el.closest(".ph-amap")) return false; + } + return true; + } + + function stops(svg) { + return Array.prototype.slice.call(svg.querySelectorAll(".hot[tabindex], .stn[tabindex]")); + } + + // Keeps the drawing's one tab stop on something that can be seen. + function refocus(svg) { + if (!svg) return; + var all = stops(svg); + var current = all.filter(function (el) { + return el.getAttribute("tabindex") === "0"; + })[0]; + if (current && isShown(current)) return; + var next = all.filter(isShown).sort(function (a, b) { + var ra = a.closest(".row"); + var rb = b.closest(".row"); + if (ra !== rb && ra && rb) return ra.compareDocumentPosition(rb) & 4 ? -1 : 1; + return box(a).x - box(b).x; + })[0]; + all.forEach(function (el) { + el.setAttribute("tabindex", el === next ? "0" : "-1"); + }); + } + + function box(el) { + var shape = el.querySelector(".g, circle, .net-line, .dot, .hub, .end") || el; + try { + return shape.getBBox(); + } catch (e) { + return { x: 0, y: 0, width: 0, height: 0 }; + } + } + + /* โ”€โ”€ The keyboard โ”€โ”€ + + Tab reaches a drawing once. Inside it, Left and Right move along a row, + Home and End go to its ends, and Up and Down go to the gene the row above + or below aligns with this one: a Bakta call, the protein placed on it, + the gene AMAP keeps. */ + + var KEYS = { ArrowLeft: 1, ArrowRight: 1, ArrowUp: 1, ArrowDown: 1, Home: 1, End: 1 }; + + function byX(list) { + return list.sort(function (a, b) { + var ba = box(a); + var bb = box(b); + return ba.x - bb.x || ba.y - bb.y; + }); + } + + function partner(from, row) { + var a = box(from); + var best = null; + var score = -Infinity; + row.forEach(function (el) { + var b = box(el); + var shared = Math.min(a.x + a.width, b.x + b.width) - Math.max(a.x, b.x); + var s = shared > 0 ? shared : -Math.abs(a.x + a.width / 2 - (b.x + b.width / 2)); + if (s > score) { + score = s; + best = el; + } + }); + return best; + } + + function moveFrom(el, key, svg) { + if (svg.classList.contains("trk")) { + var rows = Array.prototype.slice.call(svg.querySelectorAll("g.row")).filter(isShown).map(function (row) { + return byX(Array.prototype.slice.call(row.querySelectorAll(".hot[tabindex]")).filter(isShown)); + }).filter(function (row) { + return row.length; + }); + var ri = -1; + var gi = -1; + rows.forEach(function (row, i) { + var k = row.indexOf(el); + if (k !== -1) { + ri = i; + gi = k; + } + }); + if (ri === -1) return rows.length ? rows[0][0] : null; + var row = rows[ri]; + if (key === "ArrowRight") return row[Math.min(row.length - 1, gi + 1)]; + if (key === "ArrowLeft") return row[Math.max(0, gi - 1)]; + if (key === "Home") return row[0]; + if (key === "End") return row[row.length - 1]; + if (key === "ArrowDown") return ri + 1 < rows.length ? partner(el, rows[ri + 1]) : el; + if (key === "ArrowUp") return ri > 0 ? partner(el, rows[ri - 1]) : el; + return el; + } + var items = stops(svg).filter(isShown); + if (svg.classList.contains("net")) { + // The proteins, then the lines between them. + items = items.filter(function (it) { + return it.classList.contains("net-node"); + }).concat(items.filter(function (it) { + return !it.classList.contains("net-node"); + })); + } + if (svg.classList.contains("metro")) { + items = byX(items); + var here = box(el); + if (key === "ArrowUp" || key === "ArrowDown") { + var down = key === "ArrowDown"; + var pick = null; + var dist = Infinity; + items.forEach(function (other) { + var b = box(other); + var dy = b.y + b.height / 2 - (here.y + here.height / 2); + if (down ? dy < 6 : dy > -6) return; + var d = Math.abs(dy) + 2 * Math.abs(b.x + b.width / 2 - (here.x + here.width / 2)); + if (d < dist) { + dist = d; + pick = other; + } + }); + return pick || el; + } + } + var at = items.indexOf(el); + if (key === "Home") return items[0]; + if (key === "End") return items[items.length - 1]; + if (key === "ArrowRight" || key === "ArrowDown") return items[Math.min(items.length - 1, at + 1)]; + return items[Math.max(0, at - 1)]; + } + + // One listener per figure, on the element that holds its drawing. + function roving(container) { + if (container._roving) return; + container._roving = true; + container.addEventListener("keydown", function (event) { + if (!KEYS[event.key] || event.altKey || event.ctrlKey || event.metaKey) return; + var el = event.target.closest && event.target.closest(".hot[tabindex], .stn[tabindex]"); + var svg = el && el.closest("svg"); + if (!svg || !container.contains(svg)) return; + event.preventDefault(); + event.stopPropagation(); + var next = moveFrom(el, event.key, svg); + if (next && next !== el) { + stops(svg).forEach(function (other) { + other.setAttribute("tabindex", other === next ? "0" : "-1"); + }); + next.focus(); + } + }); + container.addEventListener("focusin", function (event) { + var el = event.target.closest && event.target.closest(".hot[tabindex], .stn[tabindex]"); + if (!el) return; + // A gene picked with the pointer becomes the drawing's tab stop. + stops(el.closest("svg")).forEach(function (other) { + other.setAttribute("tabindex", other === el ? "0" : "-1"); + }); + }); + } + + /* โ”€โ”€ Tooltips โ”€โ”€ + + With a mouse the tip floats over the drawing, above the gene with its + caret on it, so the rows under the gene stay in sight. On a touch + screen, or a drawing too narrow for a tip beside its rows, it is a line + of text under the drawing instead, which never covers them. Escape hides + it. */ + + // Under this width a floating tip (up to 290 px) would cover the drawing. + var NARROW = 420; + + var openTips = []; + + document.addEventListener("keydown", function (event) { + if (event.key === "Escape") { + openTips.forEach(function (hide) { + hide(true); + }); + } + }); + + function attachTips(container, opts) { + opts = opts || {}; + var what = opts.what || "a gene for its coordinates."; + var tip = container.querySelector(":scope > .demo-tip"); if (!tip) { tip = document.createElement("div"); tip.className = "demo-tip"; tip.hidden = true; container.appendChild(tip); } + var readout = container.querySelector(":scope > .fig-readout"); + if (!readout) { + readout = document.createElement("p"); + readout.className = "fig-readout"; + readout.setAttribute("aria-live", "polite"); + container.appendChild(readout); + } + var hint = 'Point atTap ' + esc(what) + ""; + readout.innerHTML = hint; + function inline() { + return mq("(hover: none)") || container.clientWidth < NARROW; + } + // A mouse moving from gene to gene must not make the page under the + // drawing jump, so there the line keeps the height of the figure's + // longest tip. A tap is deliberate: the line keeps three lines' room and + // grows for a longer tip. + var fitted = -1; + function fit() { + var w = readout.offsetWidth; + if (w === fitted) return; + fitted = w; + readout.style.minHeight = ""; + if (!w || mq("(hover: none)")) return; + var probe = readout.cloneNode(false); + probe.removeAttribute("aria-live"); + probe.setAttribute("aria-hidden", "true"); + probe.style.cssText = "position:absolute;visibility:hidden;pointer-events:none;left:0;top:0;min-height:0;width:" + w + "px"; + container.appendChild(probe); + var tallest = 0; + container.querySelectorAll(".hot[data-tip]").forEach(function (h) { + probe.innerHTML = h.getAttribute("data-tip"); + tallest = Math.max(tallest, probe.offsetHeight); + }); + probe.remove(); + if (tallest) readout.style.minHeight = tallest + "px"; + // The two networks of a pair read out side by side, so both lines keep + // the taller of their two heights and end level. + var pair = container.closest(".dup-sides"); + if (!pair) return; + container.setAttribute("data-tallest", tallest); + var most = 0; + pair.querySelectorAll("[data-tallest]").forEach(function (c) { + most = Math.max(most, +c.getAttribute("data-tallest")); + }); + if (most) { + pair.querySelectorAll(".fig-readout").forEach(function (r) { + r.style.minHeight = most + "px"; + }); + } + } + function sync() { + var on = inline(); + container.classList.toggle("tips-inline", on); + if (on) fit(); + else fitted = -1; + } + sync(); + if ("ResizeObserver" in window) new ResizeObserver(sync).observe(container); + var shownTarget = null; + // A gene reached by keyboard says which keys move on from it, in the + // words how-it-works.html uses beside its own drawing. + function keysFor(target) { + var svg = target.closest("svg"); + if (!svg || container.closest("[data-how]") || !target.matches(":focus-visible")) return ""; + var keys = svg.classList.contains("trk") + ? "← → along a row, ↑ ↓ between rows" + : "← → from one to the next"; + return '"; + } + // A tip stays while the pointer is on it, and for a moment after the + // pointer leaves a gene, so it can be read and pointed at (WCAG 1.4.13). + var tipTimer = 0; function show(target) { var text = target.getAttribute("data-tip"); if (!text) return; + text += keysFor(target); + clearTimeout(tipTimer); + sync(); container.querySelectorAll(".hot.on").forEach(function (el) { el.classList.remove("on"); }); target.classList.add("on"); + shownTarget = target; + if (inline()) { + tip.hidden = true; + readout.innerHTML = text; + return; + } tip.innerHTML = text; tip.hidden = false; - var box = container.getBoundingClientRect(); - var r = target.getBoundingClientRect(); - var left = r.left - box.left + r.width / 2 - tip.offsetWidth / 2; - left = Math.max(0, Math.min(left, box.width - tip.offsetWidth)); - var topPos = r.top - box.top - tip.offsetHeight - 8; - if (topPos < 0) topPos = r.bottom - box.top + 8; + var bounds = container.getBoundingClientRect(); + var r = (target.querySelector(".g, circle, .net-line, .net-pill") || target).getBoundingClientRect(); + var centre = r.left - bounds.left + r.width / 2; + var left = centre - tip.offsetWidth / 2; + left = Math.max(0, Math.min(left, bounds.width - tip.offsetWidth)); + // Above the gene, over the rows above it or past the drawing's top, so + // the rows under it stay in sight; below only where the window ends. + var topPos = r.top - bounds.top - tip.offsetHeight - 10; + var below = bounds.top + topPos < 72; + if (below) topPos = r.bottom - bounds.top + 10; + tip.classList.toggle("below", below); tip.style.left = left + "px"; tip.style.top = topPos + "px"; + tip.style.setProperty("--caret", Math.max(10, Math.min(tip.offsetWidth - 10, centre - left)) + "px"); } - function hide() { + function hide(clear) { tip.hidden = true; + shownTarget = null; container.querySelectorAll(".hot.on").forEach(function (el) { el.classList.remove("on"); }); + if (clear) readout.innerHTML = hint; + } + openTips.push(hide); + // The gene nearest a tap that missed every one, in the same row and + // within 16 px of it. + function nearest(event) { + var best = null; + var gap = 17; + container.querySelectorAll(".hot[data-tip]").forEach(function (h) { + if (!isShown(h)) return; + var r = (h.querySelector(".g, circle") || h).getBoundingClientRect(); + if (!r.width || event.clientY < r.top - 8 || event.clientY > r.bottom + 8) return; + var d = event.clientX < r.left ? r.left - event.clientX : event.clientX > r.right ? event.clientX - r.right : 0; + if (d < gap) { + gap = d; + best = h; + } + }); + return best; } container.addEventListener("mouseover", function (event) { + if (tip.contains(event.target)) { + clearTimeout(tipTimer); + return; + } var hot = event.target.closest(".hot"); - if (hot && container.contains(hot)) show(hot); + if (hot && container.contains(hot) && isShown(hot)) show(hot); + // Off every gene, the tip goes, unless it is the focused gene's. + else if (!inline() && shownTarget && document.activeElement !== shownTarget) { + clearTimeout(tipTimer); + tipTimer = setTimeout(hide, 200); + } + }); + container.addEventListener("mouseleave", function () { + if (!inline()) hide(); }); - container.addEventListener("mouseleave", hide); container.addEventListener("focusin", function (event) { var hot = event.target.closest(".hot"); + // A focused gene's name is read already; the line under the drawing + // would only say it again. + readout.setAttribute("aria-live", "off"); if (hot) show(hot); }); - container.addEventListener("focusout", hide); + container.addEventListener("focusout", function () { + if (!inline()) hide(); + }); container.addEventListener("click", function (event) { + if (tip.contains(event.target)) return; + // A tap says what it found, there and only there. + if (event.detail) readout.setAttribute("aria-live", "polite"); var hot = event.target.closest(".hot"); + if (!hot || !isShown(hot)) hot = nearest(event); if (hot) show(hot); - else hide(); + else hide(true); }); } + /* โ”€โ”€ Legends โ”€โ”€ + + Under each drawing, a chip for every kind of mark it uses and no other, + drawn with the same fills as the drawing. */ + + var LEGEND = [ + ["bakta", "bakta", "Bakta's call"], + ["ref", "ref", null], + ["short", "ref short", "alignment short of the whole protein"], + ["amap", "amap", "what AMAP keeps"], + ["partial", "amap partial", "partial: cut by a contig end"], + ["pseudo", "pseudo", "pseudogene"], + ["c1", "c1", "nothing aligns: case 1"], + ["drop", "drop", "dropped"], + ["bad", "drop", "a call Bakta is known to make wrongly"], + ["agree", "band-agree", "same coordinates"], + ["conflict", "band-conflict", "coordinates differ"], + ["moved", "moved", "an end AMAP moved"], + ["diff", "diff", "the stretch the text names"], + ["diff-ref", "diff-ref", "a reference gene with no call"] + ]; + + function legend(used, reference, leave) { + return LEGEND.filter(function (l) { + return used && used[l[0]] && !(leave && leave.indexOf(l[0]) !== -1); + }).map(function (l) { + var text = l[2] || (reference || "reference") + " protein, aligned"; + return '' + esc(text) + ""; + }).join(""); + } + // Draws spec into stage at the stage's width, and redraws when it changes. function render(stage, specFn, after) { var last = 0; @@ -351,10 +902,12 @@ holder.className = "fig-svg"; stage.insertBefore(holder, stage.firstChild); } - holder.innerHTML = draw(specFn(), width); - if (after) after(holder.querySelector("svg")); + var spec = specFn(); + holder.innerHTML = draw(spec, width); + if (after) after(holder.querySelector("svg"), spec); } go(); + roving(stage); if ("ResizeObserver" in window) { new ResizeObserver(go).observe(stage); } else { @@ -376,7 +929,7 @@ // Where each drawing sits and what it points out. var LOOK = { - hero: { tick: 2000, kbTicks: true, alt: "The first 12.4 kb of M. tuberculosis CDC1551: twelve Bakta calls, eight H37Rv proteins aligned, and the eight genes AMAP keeps." }, + hero: { tick: 2000, kbTicks: true, dropMark: true, alt: "The first 12.4 kb of M. tuberculosis CDC1551: twelve Bakta calls, eight H37Rv proteins aligned, and the eight genes AMAP keeps." }, case2: { from: 1960, to: 3300, tick: 500, alt: "dnaN: Bakta's call and the aligned H37Rv protein start and end at the same bases; AMAP keeps Bakta's call." }, case3: { from: 23700, to: 25600, tick: 500, alt: "fhaA: the same ends as the aligned H37Rv protein, which has six residues more; AMAP keeps Bakta's call.", gaps: { Rv0020c: { at: 24706, label: "6 residues" } } }, "case4-start": { from: -60, to: 1640, tick: 500, alt: "dnaA: Bakta's call starts at base 34, the aligned H37Rv protein at base 1; AMAP keeps the reference's start.", marks: [{ s: 1, e: 33, rows: [0, 1], label: "33 bp", labelAt: "right" }] }, @@ -434,6 +987,31 @@ return /coli/.test(win.genome) ? "K-12" : "H37Rv"; } + // Where a kept gene's ends lie against the one Bakta call it overlaps on + // its strand: the ends AMAP moved, and by how much. + function moves(a, win) { + var calls = win.bakta.filter(function (c) { + return c.st === a.st && overlap(c, a) > 30; + }); + if (calls.length !== 1) return { at: [], words: [] }; + var c = calls[0]; + var minus = a.st === "-"; + var out = { at: [], words: [] }; + // Positive: the gene starts earlier, or ends later, than the call. + var start = minus ? a.e - c.e : c.s - a.s; + var end = minus ? c.s - a.s : a.e - c.e; + if (start) { + out.at.push(minus ? a.e : a.s); + out.words.push("starts " + fmt(Math.abs(start)) + " bp " + (start > 0 ? "earlier" : "later")); + } + if (end) { + out.at.push(minus ? a.s : a.e); + out.words.push("ends " + fmt(Math.abs(end)) + " bp " + (end > 0 ? "later" : "earlier")); + } + if (out.words.length) out.words = [out.words.join(" and ") + " than Bakta's call " + short(c.tag)]; + return out; + } + function rowsFor(key) { var win = DATA.windows[key]; var look = LOOK[key] || {}; @@ -446,6 +1024,9 @@ win.amap.forEach(function (a) { kept[a.id] = a; }); + var same = function (a, b) { + return a.s === b.s && a.e === b.e && a.st === b.st; + }; var bakta = win.bakta.map(function (c) { var gone = c.case === 1 && !Object.keys(kept).some(function (id) { return id === c.tag; @@ -453,29 +1034,44 @@ var tip = "Bakta call " + short(c.tag) + (c.name ? " (" + c.name + ")" : "") + "" + (c.product ? ", " + esc(c.product) : "") + "
" + span(c) + ", " + fmt(c.e - c.s + 1) + " bp
" + CASE_SAYS[c.case]; if (c.case === 1) tip += gone ? " Dropped: " + dropReason(c, win) + "." : " Kept: nothing rules it out."; - return { s: c.s, e: c.e, st: c.st, name: c.name, tip: tip, phaseCls: c.case === 1 ? (gone ? "ph-dim c1-call" : "c1-call") : "" }; + return { + s: c.s, e: c.e, st: c.st, name: c.name, tip: tip, phaseCls: c.case === 1 ? (gone ? "ph-dim c1-call" : "c1-call") : "", + k: c.case === 2 || c.case === 3 ? "agree" : gone ? "drop" : "" + }; }); var ref = win.ref.map(function (r) { - var whole = r.complete ? "the whole protein, to its stop" : "not the whole protein"; + // The piece of a gene a contig end cut (the second pass's, in the one + // window that has one) holds the protein's last residues and lacks its + // start; any other alignment that is not the whole protein stops short + // of its end. + var whole = r.complete ? "the whole protein, to its stop" : r.piece ? "a piece: the protein's last residues, without its start" : "stops short of its end: not the whole protein"; var gap = look.gaps && look.gaps[r.prot]; return { - s: r.s, e: r.e, st: r.st, name: r.prot, gap: gap ? gap.at : null, gapLabel: gap ? gap.label : null, + s: r.s, e: r.e, st: r.st, name: r.prot, gap: gap ? gap.at : null, gapLabel: gap ? gap.label : null, short: r.complete ? "" : r.piece ? "5'" : "3'", + k: win.bakta.some(function (c) { + return same(c, r); + }) ? "agree" : win.amap.some(function (a) { + return same(a, r); + }) ? "fixed" : "", tip: "" + r.prot + (r.name !== r.prot ? " (" + r.name + ")" : "") + ", " + r.len + " residues
aligned at " + span(r) + ": " + whole + (r.num !== r.e - r.s + 1 ? "; " + fmt(r.num) + " nt aligned on " + fmt(r.e - r.s + 1) + " bp" : "") }; }); function keptGene(a) { var fromCall = win.bakta.some(function (c) { - return c.s === a.s && c.e === a.e && c.st === a.st; + return same(c, a); }); var fromRef = win.ref.some(function (r) { - return r.s === a.s && r.e === a.e && r.st === a.st; + return same(r, a); }); var how = fromCall && fromRef ? "Bakta's call, as it was" : fromCall ? "Bakta's call" : fromRef ? "the aligned protein's coordinates" : "extended to its stop codon"; - var marks = (a.pseudo ? "
Pseudogene: " + esc(a.note) : "") + (a.partial ? "
Partial at its " + a.partial + " end" : ""); + var moved = fromCall ? { at: [], words: [] } : moves(a, win); + var marks = (a.pseudo ? "
Pseudogene: " + esc(a.note) : "") + (a.partial ? "
Partial at its " + a.partial + " end" : "") + + (moved.words.length ? "
It " + moved.words[0] + "." : ""); var name = a.rvs ? a.rvs.split(",")[0] : ""; return { - s: a.s, e: a.e, st: a.st, name: name, cls: a.pseudo ? "pseudo" : null, open: a.partial || "", + s: a.s, e: a.e, st: a.st, name: name, cls: a.pseudo ? "pseudo" : null, open: a.partial || "", partial: !!a.partial, moved: moved.at, + k: fromCall && fromRef ? "agree" : fromRef ? "fixed" : "", id: a.id, tip: "" + a.id + "" + (a.rvs ? ", named " + a.rvs : ", with no reference name") + "
" + span(a) + ": " + how + marks }; } @@ -486,11 +1082,31 @@ return keptGene({ id: c.id, rvs: c.rvs, s: c.s, e: c.e, st: c.st }); }), bad: (win.bad || []).map(function (f) { - return { s: f.s, e: f.e, st: f.st, tip: "A call Bakta makes on " + reference + " itself that the curated annotation does not have, aligned here at " + span(f) + "." }; + return { s: f.s, e: f.e, st: f.st, lk: "bad", tip: "A call Bakta makes on " + reference + " itself that the curated annotation does not have, aligned here at " + span(f) + "." }; }) }; } + // The genes a figure is about (those under its marks) keep their name even + // where they are too narrow for it: it goes over them. + function nameSubjects(spec) { + (spec.marks || []).forEach(function (m) { + var lo = Math.min(m.rows[0], m.rows[1]); + var hi = Math.max(m.rows[0], m.rows[1]); + spec.rows.forEach(function (row, i) { + row.genes.forEach(function (g) { + if (!g.name) return; + // In the rows the mark spans, a gene it covers at least half of + // (or that covers half of it); in AMAP's row, the gene it frames. + var inside = i >= lo && i <= hi && overlap(g, m) >= Math.min(g.e - g.s + 1, m.e - m.s + 1) / 2; + var framed = row.cls === "amap" && g.s >= m.s && g.e <= m.e; + if (inside || framed) g.nameAbove = true; + }); + }); + }); + return spec; + } + function windowSpec(key) { var parts = rowsFor(key); var win = parts.win; @@ -535,62 +1151,127 @@ win.bakta.forEach(function (c, i) { win.ref.forEach(function (r, j) { if (r.st !== c.st || overlap(c, r) <= 30) return; - ribbons.push({ a: [0, i], b: [1, j], kind: r.s === c.s && r.e === c.e ? "agree" : "conflict", phase: key === "hero" ? "case" : null }); + var agree = r.s === c.s && r.e === c.e; + ribbons.push({ a: [0, i], b: [1, j], kind: agree ? "agree" : "conflict", k: agree ? "agree" : "fixed", phase: key === "hero" ? "case" : null }); }); }); + var marks = (look.marks || []).map(function (m) { + var named = { bakta: 0, ref: 1, amap: rows.length - 1 }; + var copy = {}; + Object.keys(m).forEach(function (k) { + copy[k] = m[k]; + }); + copy.rows = m.rows.map(function (r) { + return typeof r === "string" ? named[r] : r; + }); + return copy; + }); + if (look.dropMark) { + // The calls the hero drops, marked once AMAP has kept what it keeps. + var gone = win.bakta.filter(function (c, i) { + return bakta[i].k === "drop"; + }); + if (gone.length) { + marks.push({ + s: Math.min.apply(null, gone.map(function (c) { + return c.s; + })), + e: Math.max.apply(null, gone.map(function (c) { + return c.e; + })), + rows: [0, 0], label: gone.length + " dropped", phase: "amap" + }); + } + } var hero = key === "hero"; - return { + return nameSubjects({ from: look.from != null ? look.from : win.from, to: look.to != null ? look.to : win.to, labelW: hero ? 66 : 64, geneH: hero ? 20 : 18, rowGap: hero ? 30 : 22, - top: hero ? 6 : 18, + top: hero ? 18 : 18, alt: look.alt || "", + refName: parts.reference, + phased: true, rows: rows, ribbons: ribbons, - marks: (look.marks || []).map(function (m) { - var named = { bakta: 0, ref: 1, amap: rows.length - 1 }; - var copy = {}; - Object.keys(m).forEach(function (k) { - copy[k] = m[k]; - }); - copy.rows = m.rows.map(function (r) { - return typeof r === "string" ? named[r] : r; - }); - return copy; - }), + marks: marks, edges: look.edges || [], axis: true, tick: look.tick, tickLabel: look.kbTicks ? function (t) { return t / 1000 + " kb"; } : tickLabel(win.to - win.from) - }; + }); } + /* โ”€โ”€ The hero: the first 12.4 kb of CDC1551, in four phases โ”€โ”€ + + It plays once, when it first comes into view, and holds still while the + reader points at it. Four segments show where it is and jump to a phase; + the button pauses it, or plays it again. */ + var PHASES = [ - { label: "What Bakta called", tally: false }, - { label: "Where the H37Rv proteins align", tally: false }, - { label: "Compared, call by call", tally: true }, - { label: "What AMAP keeps", tally: true } + { label: "What Bakta called" }, + { label: "Where the H37Rv proteins align" }, + { label: "Compared, call by call" }, + { label: "What AMAP keeps" } ]; + // How long each phase stays before the next; the comparison is the densest. + var DWELL = [2100, 2100, 3200]; + + // While the page prints, a count is set at once: paper shows no frames. + var printing = false; + window.addEventListener("beforeprint", function () { + printing = true; + }); + window.addEventListener("afterprint", function () { + printing = false; + }); + + function countUp(el, n, animate) { + if (!animate || reduceMotion || printing || n < 2) { + el.textContent = String(n); + return; + } + var t0 = 0; + function frame(t) { + if (!t0) t0 = t; + var f = Math.min(1, (t - t0) / 600); + el.textContent = String(Math.round(f * n)); + if (f < 1) requestAnimationFrame(frame); + } + el.textContent = "0"; + requestAnimationFrame(frame); + } + function mountHero(fig) { var stage = fig.querySelector("[data-hero-stage]"); - var stepLabel = fig.querySelector("[data-hero-step]"); - var replay = fig.querySelector("[data-hero-replay]"); + var stepEl = fig.querySelector("[data-hero-step]"); + var button = fig.querySelector("[data-hero-replay]"); var win = DATA.windows.hero; - if (!win) return; + if (!win || !stage) return; var phase = 4; var svg = null; + var lit = null; render(stage, function () { return windowSpec("hero"); - }, function (drawn) { + }, function (drawn, spec) { svg = drawn; svg.setAttribute("data-phase", String(phase)); + if (lit) svg.setAttribute("data-light", lit); + refocus(svg); + var key = fig.querySelector(".legend"); + if (key && !key._built) { + key._built = true; + // Its one marked stretch names itself ("3 dropped"). + key.innerHTML = legend(spec._used, spec.refName, ["diff"]); + } }); attachTips(stage); + // The tally counts the window's own calls: those that agree (case 2 and 3), // the genes that took the reference's coordinates, and the calls dropped. var keptIds = {}; @@ -611,72 +1292,359 @@ }).length }; var tallies = {}; + var items = []; + // The count pressed to stay lit, or null. + var pinned = null; + function show(key) { + lit = key; + items.forEach(function (it) { + it.el.classList.toggle("lit", it.key === key); + it.el.setAttribute("aria-pressed", it.key === pinned ? "true" : "false"); + }); + if (svg) { + if (lit) svg.setAttribute("data-light", lit); + else svg.removeAttribute("data-light"); + } + } fig.querySelectorAll("[data-tally]").forEach(function (el) { tallies[el.getAttribute("data-tally")] = el; + el._on = null; + // Pointing at a count lights the genes and bands it counts; pressing + // it keeps them lit until it is pressed again. + var item = el.parentNode; + var key = el.getAttribute("data-tally"); + items.push({ el: item, key: key }); + item.setAttribute("tabindex", "0"); + item.setAttribute("role", "button"); + item.setAttribute("aria-pressed", "false"); + var toggle = function () { + // A count not made yet is made first: the drawing goes to the phase + // that makes it, as pressing that phase would, and the press then + // keeps lit what the count names. + if (item.classList.contains("t-off")) { + spoke(); + played = true; + stopAuto(); + setPhase(key === "agree" ? 3 : 4); + pinned = key; + } else { + pinned = pinned === key ? null : key; + } + show(pinned || key); + }; + item.addEventListener("pointerenter", function () { + show(key); + }); + item.addEventListener("pointerleave", function () { + show(pinned); + }); + item.addEventListener("focus", function () { + show(key); + }); + item.addEventListener("blur", function () { + show(pinned); + }); + item.addEventListener("click", toggle); + item.addEventListener("keydown", function (event) { + if (event.key !== "Enter" && event.key !== " ") return; + event.preventDefault(); + toggle(); + }); }); + + // The four segments, built into the step line, and its words. + var segs = []; + var words = null; + if (stepEl) { + stepEl.setAttribute("aria-live", "off"); + stepEl.textContent = ""; + var group = document.createElement("span"); + group.className = "hero-segs"; + group.setAttribute("role", "group"); + group.setAttribute("aria-label", "The four phases"); + PHASES.forEach(function (p, i) { + var b = document.createElement("button"); + b.type = "button"; + b.className = "hero-seg"; + b.setAttribute("data-go", String(i + 1)); + b.setAttribute("aria-label", "Phase " + (i + 1) + " of 4: " + p.label); + b.innerHTML = ""; + group.appendChild(b); + segs.push(b); + }); + words = document.createElement("span"); + words.className = "hero-phase"; + stepEl.appendChild(group); + stepEl.appendChild(words); + } + + // The line keeps the height of its longest phase, so a phase whose words + // wrap at this width does not push the drawing down while it plays. + var stepWidth = 0; + function fitStep() { + if (!stepEl || !words) return; + var w = stepEl.offsetWidth; + if (!w || Math.abs(w - stepWidth) < 2) return; + stepWidth = w; + var probe = stepEl.cloneNode(true); + probe.removeAttribute("aria-live"); + probe.removeAttribute("data-hero-step"); + probe.setAttribute("aria-hidden", "true"); + probe.style.cssText = "position:absolute;visibility:hidden;pointer-events:none;left:0;top:0;min-height:0;width:" + w + "px"; + stepEl.parentNode.appendChild(probe); + var said = probe.querySelector(".hero-phase"); + var tallest = 0; + PHASES.forEach(function (p, i) { + said.textContent = i + 1 + " of 4 ยท " + p.label; + tallest = Math.max(tallest, probe.offsetHeight); + }); + probe.remove(); + stepEl.style.minHeight = tallest + "px"; + } + fitStep(); + if (stepEl && "ResizeObserver" in window) new ResizeObserver(fitStep).observe(stepEl); + + var auto = false; + var userPaused = false; + var hover = false; + var focus = false; var timer = null; + var started = 0; + var left = 0; + var played = false; + var inView = false; + + // After the reader's first move, and not before, the phases are spoken. + function spoke() { + if (stepEl) stepEl.setAttribute("aria-live", "polite"); + } + function setPhase(n) { phase = n; - if (svg) svg.setAttribute("data-phase", String(n)); - stepLabel.textContent = n + " of 4 ยท " + PHASES[n - 1].label; - var on = PHASES[n - 1].tally; + if (svg) { + svg.setAttribute("data-phase", String(n)); + refocus(svg); + } + if (words) words.textContent = n + " of 4 ยท " + PHASES[n - 1].label; + segs.forEach(function (b, i) { + b.classList.toggle("done", i + 1 < n); + b.classList.remove("now"); + if (i + 1 === n) { + // Reflow, so the fill starts again from empty. + void b.offsetWidth; + b.classList.add("now"); + b.style.setProperty("--dwell", (DWELL[i] || 0) + "ms"); + b.setAttribute("aria-current", "step"); + } else { + b.removeAttribute("aria-current"); + } + }); + // 'agree' once the calls are compared; the rest once AMAP has kept. Object.keys(tallies).forEach(function (k) { - tallies[k].textContent = String(counts[k]); - tallies[k].parentNode.classList.toggle("t-off", !on); + var on = k === "agree" ? n >= 3 : n >= 4; + var el = tallies[k]; + el.parentNode.classList.toggle("t-off", !on); + // A count not yet made shows no number, so nothing reads as a count + // that is not true yet. + if (on && el._on !== true) countUp(el, counts[k], el._on !== null); + else if (!on) el.textContent = "\u2013"; + if (on) el.removeAttribute("aria-hidden"); + else el.setAttribute("aria-hidden", "true"); + el._on = on; }); } - function play() { - clearTimeout(timer); - if (reduceMotion) { - setPhase(4); + + function held() { + return hover || focus || userPaused; + } + function arm(ms) { + left = ms; + if (held()) { + fig.classList.add("paused"); return; } - var n = 1; - setPhase(n); - (function next() { - timer = setTimeout(function () { - n += 1; - setPhase(n); - if (n < 4) next(); - }, 2100); - })(); + started = Date.now(); + timer = setTimeout(next, ms); + } + function next() { + timer = null; + setPhase(phase + 1); + if (phase < 4) arm(DWELL[phase - 1]); + else finish(); + } + function finish() { + auto = false; + userPaused = false; + fig.classList.remove("running", "paused"); + label(); + } + function freeze() { + if (timer) { + clearTimeout(timer); + timer = null; + left = Math.max(0, left - (Date.now() - started)); + } + if (auto) fig.classList.add("paused"); + } + function thaw() { + if (!auto || held()) return; + fig.classList.remove("paused"); + if (!timer) { + started = Date.now(); + timer = setTimeout(next, left); + } + } + function start(from) { + clearTimeout(timer); + timer = null; + auto = true; + played = true; + userPaused = false; + fig.classList.remove("paused"); + fig.classList.add("running"); + setPhase(from); + arm(DWELL[from - 1]); + label(); + } + function stopAuto() { + clearTimeout(timer); + timer = null; + auto = false; + userPaused = false; + fig.classList.remove("running", "paused"); + label(); + } + function label() { + if (!button) return; + var playing = auto && !userPaused; + // Both words stay in the button and one shows, so it keeps the width + // of the longer and the line beside it never rewraps. + if (!button.querySelector(".say-2")) button.innerHTML = 'PlayPause'; + button.classList.toggle("said-2", playing); + button.setAttribute("aria-label", playing ? "Pause the drawing" : "Play the drawing"); + } + + if (button) { + button.addEventListener("click", function () { + spoke(); + if (auto && !userPaused) { + userPaused = true; + freeze(); + } else if (auto) { + userPaused = false; + thaw(); + } else { + start(phase < 4 ? phase : 1); + } + label(); + }); + } + segs.forEach(function (b) { + b.addEventListener("click", function () { + spoke(); + played = true; + stopAuto(); + setPhase(+b.getAttribute("data-go")); + }); + }); + // On paper the drawing is its end, every layer in, not a moment of it. + window.addEventListener("beforeprint", function () { + stopAuto(); + setPhase(4); + }); + // Still while the reader points at the drawing or moves through it. + stage.addEventListener("pointerenter", function (event) { + if (event.pointerType !== "mouse") return; + hover = true; + freeze(); + }); + stage.addEventListener("pointerleave", function (event) { + if (event.pointerType !== "mouse") return; + hover = false; + thaw(); + }); + stage.addEventListener("focusin", function () { + focus = true; + freeze(); + }); + stage.addEventListener("focusout", function (event) { + if (event.relatedTarget && stage.contains(event.relatedTarget)) return; + focus = false; + thaw(); + }); + + // It plays when it first comes into view, however late. A page that is + // not being shown (a background tab) shows the last phase meanwhile. + function maybeStart() { + if (!played && inView && !document.hidden) start(1); } - replay.addEventListener("click", play); - // Start when the drawing comes into view, so the reader sees it happen. if ("IntersectionObserver" in window && !reduceMotion) { - setPhase(1); - var seen = false; + setPhase(document.hidden ? 4 : 1); + label(); var io = new IntersectionObserver(function (entries) { - if (!seen && entries[0].isIntersecting) { - seen = true; - io.disconnect(); - play(); - } + inView = entries[entries.length - 1].isIntersecting; + maybeStart(); + if (played) io.disconnect(); }, { threshold: 0.35 }); io.observe(fig); - // A page that is never shown (a hidden tab) still ends up drawn. - setTimeout(function () { - if (!seen) { - seen = true; - io.disconnect(); + document.addEventListener("visibilitychange", function () { + if (played) return; + if (document.hidden) { setPhase(4); + } else { + setPhase(1); + maybeStart(); } - }, 9000); + }); } else { setPhase(4); + label(); } } /* โ”€โ”€ One drawing per case โ”€โ”€ */ + // The legend under a figure of the docs: a chip for each kind of mark its + // drawing uses. + function legendAfter(stage) { + var el = document.createElement("div"); + el.className = "legend fig-legend"; + stage.parentNode.insertBefore(el, stage.nextSibling); + return el; + } + function mountCase(el) { var names = el.getAttribute("data-fig").split(" "); var stage = el.querySelector("[data-fig-stage]") || el; var switcher = el.querySelector("[data-fig-switch]"); var caption = el.querySelector("[data-fig-caption]"); var current = names[0]; + // Every drawing the switcher can show, so the figure keeps the height of + // the tallest and nothing under it moves on a switch. + var variants = switcher ? Array.prototype.map.call(switcher.querySelectorAll("button[data-show]"), function (b) { + return b.getAttribute("data-show"); + }).filter(function (n) { + return DATA.windows[n]; + }) : []; + var key = el.tagName === "FIGURE" && stage !== el ? legendAfter(stage) : null; var redraw = render(stage, function () { - return windowSpec(current); + var spec = windowSpec(current); + spec.final = true; + return spec; + }, function (svg, spec) { + // A figure shows the end of the story: what AMAP kept and dropped. + svg.setAttribute("data-phase", "4"); + if (variants.length > 1) { + var w = svg.viewBox.baseVal.width; + var tallest = 0; + variants.forEach(function (n) { + var s = windowSpec(n); + draw(s, w); + tallest = Math.max(tallest, s._h); + }); + svg.parentNode.style.minHeight = (tallest * svg.getBoundingClientRect().width) / w + "px"; + } + if (key) key.innerHTML = legend(spec._used, spec.refName); }); attachTips(stage); function show(name) { @@ -734,7 +1702,7 @@ MINIPROT: { name: "miniprot", proc: "MINIPROT", x: 440, y: 170, label: "below", hub: true, text: "miniprot aligns the reference proteome onto each genome, plus the false-positive set, plus Bakta's proteins back onto the reference genome; then, for each placement a contig end cut, what it lacks, to find the gene's other piece.", - reads: "each genome, Bakta's proteins, the reference", writes: "nothing published: the alignments go on to Classify", flags: "", doc: "how-it-works/workflow.html#4-protein-alignment-with-miniprot" + reads: "each genome, Bakta's proteins, the reference", writes: "nothing published: the alignments go on to Cases", flags: "", doc: "how-it-works/workflow.html#4-protein-alignment-with-miniprot" }, TYPE_CASES: { name: "Cases", proc: "TYPE_CASES", x: 535, y: 170, label: "above", hub: true, @@ -748,13 +1716,13 @@ }, EGGNOG_DB: { name: "eggNOG database", proc: "EGGNOG_DB", x: 735, y: 60, kind: "end", label: "left", - text: "Builds the eggNOG database on the first run, a bacteria-only index of about 3 GB plus the metadata it downloads, and reuses it after.", + text: "Builds the eggNOG database on the first run: eggnog.db, 41.4 GB unpacked from a 6.8 GB download, and a bacteria-only Diamond index of about 3 GB. It reuses them after.", reads: "", writes: "the folder --eggnog_db points at", flags: "--eggnog_db; skipped with --skip_functional_annotation", doc: "how-it-works/workflow.html#1-eggnog-database-setup", functional: true }, ANNOT_EGGNOGMAP_BATCH: { - name: "eggNOG-mapper", proc: "ANNOT_EGGNOGMAP_BATCH", x: 735, y: 170, label: "below", hub: true, hubUp: 9, hubDown: 27, + name: "eggNOG-mapper", proc: "ANNOT_EGGNOGMAP_BATCH", x: 735, y: 170, label: "below", labelDx: -10, hub: true, hubUp: 9, hubDown: 27, text: "eggNOG-mapper annotates the corrected proteins of all samples in one go: COG, KEGG, GO, EC, PFAM, CAZy.", - reads: "every sample's corrected proteins", writes: "nothing published: the annotations go on", flags: "--skip_functional_annotation stops before this", doc: "how-it-works/workflow.html#7-functional-annotation", functional: true + reads: "every sample's corrected proteins", writes: "nothing published: the annotations go on to GFF3, GenBank", flags: "--skip_functional_annotation stops before this", doc: "how-it-works/workflow.html#7-functional-annotation", functional: true }, GET_OUTPUTS: { name: "GFF3, GenBank", proc: "GET_OUTPUTS", x: 860, y: 170, label: "above", @@ -767,12 +1735,12 @@ reads: "the master table", writes: "AMAP/: *-AMAP.COG.tsv, *-AMAP.MycoBrowser.tsv", flags: "", doc: "user-guide/outputs.html#the-full-tree", functional: true }, GET_DUPLICATIONS: { - name: "Duplications", proc: "GET_DUPLICATIONS", x: 890, y: 250, label: "above", lineOnly: "dup", + name: "Duplications", proc: "GET_DUPLICATIONS", x: 890, y: 250, label: "below", labelDx: -6, lineOnly: "dup", text: "Each proteome against itself with Diamond: duplicated genes grouped into families.", - reads: "the corrected proteins and their functions", writes: "DUPLICATIONS/{sample}/", flags: "", doc: "how-it-works/duplications.html", exit: "DUPLICATIONS/", exitSide: "below", functional: true + reads: "the corrected proteins and their functions", writes: "DUPLICATIONS/{lineage}_{sampleId}/", flags: "", doc: "how-it-works/duplications.html", exit: "DUPLICATIONS/", exitSide: "below", functional: true }, GET_ALLDUP: { - name: "All genomes", proc: "GET_ALLDUP", x: 980, y: 250, label: "above", lineOnly: "dup", + name: "All genomes", proc: "GET_ALLDUP", x: 980, y: 250, label: "below", labelDx: 10, lineOnly: "dup", text: "All samples' duplicated genes against each other, as communities with network metrics.", reads: "every sample's duplicated genes", writes: "DUPLICATIONS/homologous_communities.tsv, .html", flags: "", doc: "how-it-works/duplications.html", functional: true }, @@ -782,12 +1750,12 @@ reads: "the reference proteome, the corrected genes, and every genome: as PATHOTYPR renamed it, or with --skip_typing as the sample sheet gives it, under its sample ID", writes: "PANGENE/pangenome.gfa, .bubbles.txt, .Rtab, .stat", flags: "", doc: "how-it-works/workflow.html#9-pangenome", exit: "PANGENE/", exitSide: "right", functional: true }, FINAL_REPORT: { - name: "Report", proc: "FINAL_REPORT", x: 1050, y: 170, kind: "hub-end", label: "above", + name: "Full report", proc: "FINAL_REPORT", x: 1050, y: 170, kind: "hub-end", label: "above", labelDx: -16, text: "One self-contained HTML file from everything above: quality, cases, missing genes, functions, duplications, pangenome.", reads: "every summary (REPORT_INPUTS gathers them)", writes: "final_report.html", flags: "", doc: "user-guide/report.html", exit: "final_report.html", exitSide: "below", functional: true }, AMAP_REPORT: { - name: "Interactive report", proc: "AMAP_REPORT", x: 1050, y: 100, kind: "hub-end", label: "above-end", + name: "Report by question", proc: "AMAP_REPORT", x: 1050, y: 100, kind: "hub-end", label: "above-end", text: "One self-contained HTML file organised by question: how each genome annotated, a map of the reference genes in every genome, one gene in every genome, what the correction changed, the pseudogenes by lineage. It needs only the correction, so it is written with or without eggNOG, whose steps add their own sections.", reads: "every genome's corrected genes, case tables and Bakta folder; the master tables, the pangenome and the communities when the run has them", writes: "amap_report.html", flags: "", doc: "user-guide/amap-report.html", exit: "amap_report.html", exitSide: "above" }, @@ -806,8 +1774,12 @@ "AMAP_REPORT" ]; + // The sample's line runs on past Correction only to the steps that need + // eggNOG, so it has a stretch of its own that a run without them leaves + // out. var LINES = [ - { key: "main", d: "M70,170 H1050" }, + { key: "main", d: "M70,170 H625" }, + { key: "fn", d: "M625,170 H1050" }, { key: "ref", d: "M395,60 H400 Q410,60 410,70 V151 Q410,161 420,161 H650", endTick: [650, 161] }, { key: "db", d: "M735,60 V160" }, { key: "dup", d: "M735,179 H800 Q820,179 826,200 L836,230 Q842,250 862,250 H1000 Q1020,250 1028,230 L1040,195 Q1046,179 1050,179" }, @@ -816,33 +1788,30 @@ // The interactive report reads the correction's outputs directly, and the // functional steps' when they run (the dashed stretch). { key: "rep", d: "M625,152 V118 Q625,100 643,100 H1050" }, - { key: "rep-feed", d: "M1050,158 V114", dash: true } + { key: "rep-feed", d: "M1062,170 Q1080,170 1080,152 V118 Q1080,100 1062,100", dash: true } ]; + // The map is drawn once; the switches and the selection then only change + // classes, so lines and stations fade rather than blink. function metroSvg(state) { var out = []; - out.push(''); + out.push(''); LINES.forEach(function (l) { - var off = lineOff(l.key, state); if (l.feed) { // Every genome, for the pangenome: renamed by Lineage, or from the // sample sheet when typing is off (PANGENE_GENOME names them). - var untyped = state.skipTyping; - out.push(''); - out.push('' + (untyped ? "the genomes of the sample sheet, under their sample IDs" : "the renamed genomes, from Lineage") + ""); + out.push(''); + out.push('the renamed genomes, from Lineage'); return; } - out.push('"); + out.push('"); if (l.endTick) { - out.push(''); + out.push(''); } }); - Object.keys(STATIONS).forEach(function (key) { + Object.keys(STATIONS).forEach(function (key, n) { var s = STATIONS[key]; - var off = stationOff(key, state); - var skipped = key === "PATHOTYPR" && state.skipTyping; - var cls = "stn" + (off ? " off" : "") + (skipped ? " skipped" : "") + (state.selected === key ? " on" : ""); - out.push(''); + out.push(''); if (s.hub) { // A hub spans every track that runs through it. var top = s.y - (s.hubUp == null ? 18 : s.hubUp); @@ -857,8 +1826,8 @@ } else { out.push(''); } - var proc = state.custom && s.procCustom ? s.procCustom : s.proc; - var lx = s.x; + // labelDx nudges a name clear of its neighbour's. + var lx = s.x + (s.labelDx || 0); var anchor = "middle"; var ny; var py; @@ -894,17 +1863,42 @@ py += s.hubDown - 9; } out.push('' + esc(s.name) + ""); - out.push('' + esc(proc) + ""); + out.push('' + esc(s.proc) + ""); out.push(""); }); out.push(""); return out.join(""); } + function paintMetro(svg, state) { + svg.querySelectorAll("[data-line]").forEach(function (el) { + el.classList.toggle("off", lineOff(el.getAttribute("data-line"), state)); + }); + var feed = LINES.filter(function (l) { + return l.feed; + })[0]; + var untyped = state.skipTyping; + svg.querySelector('path[data-line="pan-feed"]').setAttribute("d", untyped ? feed.dUntyped : feed.d); + var feedText = svg.querySelector("[data-feed]"); + feedText.setAttribute("x", untyped ? "34" : "270"); + feedText.textContent = untyped ? "the genomes of the sample sheet, under their sample IDs" : "the renamed genomes, from Lineage"; + svg.querySelectorAll(".stn").forEach(function (g) { + var key = g.getAttribute("data-stn"); + var s = STATIONS[key]; + var proc = state.custom && s.procCustom ? s.procCustom : s.proc; + g.classList.toggle("off", stationOff(key, state)); + g.classList.toggle("skipped", key === "PATHOTYPR" && state.skipTyping); + g.classList.toggle("on", state.selected === key); + g.setAttribute("aria-pressed", state.selected === key ? "true" : "false"); + g.setAttribute("aria-label", s.name + ", " + proc + (stationOff(key, state) ? ", not run" : "")); + g.querySelector("text.proc").textContent = proc; + }); + } + function lineOff(key, state) { if (key === "main") return false; if (key === "ref") return false; - if (state.skipFunctional && (key === "db" || key === "dup" || key === "pan" || key === "pan-feed" || key === "rep-feed")) return true; + if (state.skipFunctional && (key === "fn" || key === "db" || key === "dup" || key === "pan" || key === "pan-feed" || key === "rep-feed")) return true; return false; } @@ -924,66 +1918,156 @@ return dirs; } + function infoHtml(key, state) { + if (!key) { + // Reached by keyboard, it says how to move along the map. + return state.keys + ? '

Move along the stations with the arrow keys; Enter shows what one does, what it reads and where its files go. The switches above show what a run leaves out.

' + : '

Pick a station to see what it does, what it reads and where its files go. The switches above show what a run leaves out.

'; + } + var s = STATIONS[key]; + var proc = state.custom && s.procCustom ? s.procCustom : s.proc; + var text = state.custom && s.textCustom ? s.textCustom : s.text; + var status = ""; + if (stationOff(key, state)) status = '

Not run with the switches above.

'; + if (key === "PANGENE" && state.skipTyping && !state.skipFunctional) status = '

With --skip_typing it searches the genomes of the sample sheet, each under its sample ID.

'; + return ( + "

" + esc(s.name) + "" + esc(proc) + "

" + esc(tie(text)) + "

" + status + + (s.doc ? '

More on this step

' : "") + "
" + + "
" + + (s.reads ? "
Reads
" + esc(s.reads) + "
" : "") + + (s.writes ? "
Writes
" + esc(s.writes) + "
" : "") + + (s.flags ? "
Options
" + esc(s.flags) + "
" : "") + + "
" + ); + } + function mountMetro(el) { var stage = el.querySelector("[data-metro-stage]"); var info = el.querySelector("[data-metro-info]"); var writesEl = el.querySelector("[data-metro-writes]"); + // The line chips of the legend, where the page has them, by their line. + var chips = Array.prototype.map.call(el.querySelectorAll(".metro-lines span"), function (span) { + var i = span.querySelector("i"); + var m = /--m-(\w+)/.exec((i && i.getAttribute("style")) || ""); + return { span: span, line: m ? m[1] : "" }; + }); var state = { skipTyping: false, skipFunctional: false, custom: false, selected: null }; - function render() { - stage.innerHTML = metroSvg(state); + stage.innerHTML = metroSvg(state); + var svg = stage.querySelector("svg"); + roving(stage); + + // Once a station is picked, the panel under the map keeps the height of + // the tallest station's text, so picking another never moves what + // follows. Before that it holds only its short hint. + function fit() { + if (!info || !info.offsetWidth) return; + info.style.minHeight = ""; + if (!state.selected) return; + var probe = info.cloneNode(false); + probe.removeAttribute("data-metro-info"); + probe.removeAttribute("aria-live"); + probe.setAttribute("aria-hidden", "true"); + probe.style.cssText = "position:absolute;visibility:hidden;pointer-events:none;left:0;top:0;min-height:0;width:" + info.offsetWidth + "px"; + info.parentNode.appendChild(probe); + var tallest = 0; + Object.keys(STATIONS).forEach(function (key) { + probe.innerHTML = infoHtml(key, state); + tallest = Math.max(tallest, probe.offsetHeight); + }); + probe.remove(); + var kept = parseFloat(getComputedStyle(info).minHeight) || 0; + info.style.minHeight = Math.max(kept, tallest) + "px"; + } + var fitWidth = 0; + if (info && "ResizeObserver" in window) { + new ResizeObserver(function () { + if (Math.abs(info.offsetWidth - fitWidth) < 2) return; + fitWidth = info.offsetWidth; + fit(); + }).observe(info); + } + + // On a narrow screen the map scrolls sideways; a fade at the edge says + // there is more of it. + function edges() { + var more = stage.scrollWidth - stage.clientWidth; + stage.classList.toggle("more-right", more > 2 && stage.scrollLeft < more - 2); + stage.classList.toggle("more-left", more > 2 && stage.scrollLeft > 2); + } + stage.addEventListener("scroll", edges, { passive: true }); + if ("ResizeObserver" in window) new ResizeObserver(edges).observe(stage); + + function paint() { + paintMetro(svg, state); + chips.forEach(function (c) { + c.span.classList.toggle("off", state.skipFunctional && (c.line === "db" || c.line === "dup" || c.line === "pan")); + }); if (writesEl) { writesEl.innerHTML = "This run writes " + writes(state).map(function (d) { return "" + d + ""; }).join(" ") + "."; } describe(); + edges(); } + // The panel is a live region: it is only rewritten when what it says + // changes, so a switch that changes nothing there is not read out again. function describe() { if (!info) return; - var key = state.selected; - if (!key) { - info.innerHTML = '

Pick a station to see what it does, what it reads and where its files go. The switches above show what a run leaves out.

'; - return; + var html = infoHtml(state.selected, state); + if (html !== info._said) { + info._said = html; + info.innerHTML = html; + } + } + function pick(g) { + var key = g.getAttribute("data-stn"); + if (key === state.selected) return; + var first = !state.selected; + state.selected = key; + if (first) fit(); + paint(); + // The station picked comes to the middle of a map that scrolls. + if (stage.scrollWidth > stage.clientWidth + 2) { + var r = g.getBoundingClientRect(); + var sr = stage.getBoundingClientRect(); + var by = r.left + r.width / 2 - (sr.left + sr.width / 2); + if (Math.abs(by) > 8) stage.scrollBy({ left: by, behavior: reduceMotion ? "auto" : "smooth" }); } - var s = STATIONS[key]; - var proc = state.custom && s.procCustom ? s.procCustom : s.proc; - var text = state.custom && s.textCustom ? s.textCustom : s.text; - var status = ""; - if (stationOff(key, state)) status = '

Not run with the switches above.

'; - if (key === "PANGENE" && state.skipTyping && !state.skipFunctional) status = '

With --skip_typing it searches the genomes of the sample sheet, each under its sample ID.

'; - info.innerHTML = - "

" + esc(s.name) + "" + esc(proc) + "

" + esc(text) + "

" + status + - (s.doc ? '

More on this step

' : "") + "
" + - "
" + - (s.reads ? "
Reads
" + esc(s.reads) + "
" : "") + - (s.writes ? "
Writes
" + esc(s.writes) + "
" : "") + - (s.flags ? "
Options
" + esc(s.flags) + "
" : "") + - "
"; } el.addEventListener("change", function (event) { var t = event.target; if (t.matches("[data-metro-opt]")) { state[t.getAttribute("data-metro-opt")] = t.checked; - render(); + paint(); + fit(); } }); stage.addEventListener("click", function (event) { var g = event.target.closest(".stn"); - if (!g) return; - state.selected = g.getAttribute("data-stn"); - render(); + if (g) pick(g); + }); + stage.addEventListener("focusin", function (event) { + var g = event.target.closest && event.target.closest(".stn"); + if (!g || state.keys || !g.matches(":focus-visible")) return; + state.keys = true; + describe(); + }); + stage.addEventListener("focusout", function (event) { + if (!state.keys || (event.relatedTarget && stage.contains(event.relatedTarget))) return; + state.keys = false; + describe(); }); stage.addEventListener("keydown", function (event) { if (event.key !== "Enter" && event.key !== " ") return; var g = event.target.closest(".stn"); if (!g) return; event.preventDefault(); - state.selected = g.getAttribute("data-stn"); - render(); - var again = stage.querySelector('[data-stn="' + state.selected + '"]'); - if (again) again.focus(); + pick(g); }); - render(); + paint(); + fit(); } /* โ”€โ”€ The overview: what goes into a decision, and what comes out โ”€โ”€ */ @@ -994,8 +2078,9 @@ function inputsSpec() { var parts = rowsFor("hero"); return { - from: parts.win.from, to: parts.win.to, labelW: 84, geneH: 18, rowGap: 22, top: 8, + from: parts.win.from, to: parts.win.to, labelW: 84, geneH: 18, rowGap: 22, top: 18, alt: "The first 12.4 kb of M. tuberculosis CDC1551: Bakta's twelve calls, the eight H37Rv proteins aligned, and two of Bakta's known false positives.", + refName: parts.reference, rows: [ { label: "Bakta's\ncalls", cls: "bakta", genes: parts.bakta }, { label: parts.reference + "\nproteins", cls: "ref", genes: parts.ref }, @@ -1019,6 +2104,9 @@ } // A small drawing of one window: its rows without their names, no axis. + // It sits inside a link, so it is a picture: no tab stops, no tips. The + // gene it is about keeps its name, beside it where it is too narrow for + // it (the gene put back, in the card of the missing gene). function miniSpec(key) { var spec = windowSpec(key); spec.labelW = 0; @@ -1027,12 +2115,14 @@ spec.rowGap = 12; spec.top = spec.marks.length ? 16 : 4; spec.padRight = 2; + spec.inert = true; spec.marks.forEach(function (m) { m.label = ""; }); spec.rows.forEach(function (row) { row.genes.forEach(function (g) { g.gapLabel = null; + g.moved = []; }); }); spec.edges = []; @@ -1042,7 +2132,10 @@ function mountInputs(el) { var stage = el.querySelector("[data-fig-stage]"); if (!DATA.windows.hero) return; - render(stage, inputsSpec); + var key = legendAfter(stage); + render(stage, inputsSpec, function (svg, spec) { + key.innerHTML = legend(spec._used, spec.refName); + }); attachTips(stage); } @@ -1053,6 +2146,8 @@ if (!DATA.windows[key]) return; render(stage, function () { return miniSpec(key); + }, function (svg) { + svg.setAttribute("data-phase", "4"); }); }); } @@ -1066,6 +2161,11 @@ var DUP = window.AMAP_DUP_DATA || null; + // A genome's name, its species in italics. + function speciesHtml(label) { + return esc(label).replace(/^([A-Z])\. ([a-z]+)(?= |$)/, "$1.\u00a0$2"); + } + function pct(x) { return (Math.round(x * 10) / 10).toFixed(1) + "%"; } @@ -1076,7 +2176,8 @@ } // A network: nodes at x, y between 0 and 1, links between them, and a soft - // outline around the nodes of a family. + // outline around the nodes of a family. One tab stop, like a gene track: + // the arrows move through the proteins, then the lines. function netSvg(o, width) { var W = Math.max(260, Math.round(width)); var H = Math.round(W < 520 ? W * (o.tall || 0.92) : Math.min(o.maxH || 380, W * (o.ratio || 0.56))); @@ -1087,7 +2188,7 @@ var Y = function (y) { return pad - 6 + y * (H - 2 * pad); }; - var out = ['']; + var out = ['']; if (o.hull && o.hull.length) { var pts = o.hull.map(function (i) { return X(o.nodes[i].x).toFixed(1) + "," + Y(o.nodes[i].y).toFixed(1); @@ -1104,11 +2205,14 @@ return b.x + b.w < c.x || c.x + c.w < b.x || b.y + b.h < c.y || c.y + c.h < b.y; }); }; + var item = function (tip) { + return ' role="img" tabindex="-1" aria-label="' + esc(plain(tip)) + '" data-tip="' + esc(tip) + '"'; + }; o.links.forEach(function (l) { var a = o.nodes[l.a]; var b = o.nodes[l.b]; out.push( - '' + + '" + '' + '' ); @@ -1132,20 +2236,88 @@ }); if (!spot) return; boxes.push(spot); + // The label repeats its line, which says it all to a screen reader. out.push( - '' + + '" ); }); - o.nodes.forEach(function (n) { + // Each name takes the place round its node that crosses the fewest + // lines (a solid link counts double), never another node, a label or the + // frame's edge; on a tie, the first of: under, over, right, left, the + // corners under it, then under and over to either side. + var segs = o.links.map(function (l) { + return [X(o.nodes[l.a].x), Y(o.nodes[l.a].y), X(o.nodes[l.b].x), Y(o.nodes[l.b].y), l.cls === "kept" ? 2 : 1]; + }); + var crossings = function (b) { + var n = 0; + segs.forEach(function (s) { + var len = Math.hypot(s[2] - s[0], s[3] - s[1]); + for (var d = 0; d <= len; d += 2) { + var px = s[0] + ((s[2] - s[0]) * d) / (len || 1); + var py = s[1] + ((s[3] - s[1]) * d) / (len || 1); + if (px > b.x - 3 && px < b.x + b.w + 3 && py > b.y - 3 && py < b.y + b.h + 3) { + n += s[4]; + return; + } + } + }); + return n; + }; + var taken = boxes.filter(function (b) { + return b.h === 16; + }); + var cost = function (b, self) { + if (b.x < 2 || b.y < 2 || b.x + b.w > W - 2 || b.y + b.h > H - 2) return Infinity; + var hitsNode = o.nodes.some(function (m, j) { + if (j === self) return false; + var mx = X(m.x); + var my = Y(m.y); + var dx = Math.max(b.x - mx, 0, mx - (b.x + b.w)); + var dy = Math.max(b.y - my, 0, my - (b.y + b.h)); + return Math.hypot(dx, dy) < m.r + 3; + }); + var hitsLabel = !taken.every(function (c) { + return b.x + b.w < c.x || c.x + c.w < b.x || b.y + b.h < c.y || c.y + c.h < b.y; + }); + return hitsNode || hitsLabel ? Infinity : crossings(b); + }; + o.nodes.forEach(function (n, i) { var cx = X(n.x); var cy = Y(n.y); - // The name goes under the node, or over it at the bottom of the frame. - var below = cy + n.r + 14 < H - 2; - out.push(''); + var tw = textWidth(n.label, 11.5) * 1.05; + var under = { x: cx - tw / 2, y: cy + n.r + 2, w: tw, h: 14, tx: cx, ty: cy + n.r + 13, a: "middle" }; + var over = { x: cx - tw / 2, y: cy - n.r - 17, w: tw, h: 14, tx: cx, ty: cy - n.r - 6, a: "middle" }; + var spots = [ + under, + over, + { x: cx + n.r + 4, y: cy - 7, w: tw, h: 14, tx: cx + n.r + 5, ty: cy + 4, a: "start" }, + { x: cx - n.r - 4 - tw, y: cy - 7, w: tw, h: 14, tx: cx - n.r - 5, ty: cy + 4, a: "end" }, + { x: cx + n.r * 0.7 + 2, y: cy + n.r * 0.7, w: tw, h: 14, tx: cx + n.r * 0.7 + 3, ty: cy + n.r * 0.7 + 11, a: "start" }, + { x: cx - n.r * 0.7 - 2 - tw, y: cy + n.r * 0.7, w: tw, h: 14, tx: cx - n.r * 0.7 - 3, ty: cy + n.r * 0.7 + 11, a: "end" }, + { x: cx + 2, y: cy + n.r + 2, w: tw, h: 14, tx: cx + 2, ty: cy + n.r + 13, a: "start" }, + { x: cx - 2 - tw, y: cy + n.r + 2, w: tw, h: 14, tx: cx - 2, ty: cy + n.r + 13, a: "end" }, + { x: cx + 2, y: cy - n.r - 17, w: tw, h: 14, tx: cx + 2, ty: cy - n.r - 6, a: "start" }, + { x: cx - 2 - tw, y: cy - n.r - 17, w: tw, h: 14, tx: cx - 2, ty: cy - n.r - 6, a: "end" } + ]; + var spot = null; + var best = Infinity; + spots.forEach(function (b) { + var c = cost(b, i); + if (c < best) { + best = c; + spot = b; + } + }); + if (!spot) spot = cy + n.r + 14 < H - 2 ? under : over; + taken.push(spot); + out.push('"); + out.push(''); out.push(''); - out.push('' + esc(n.label) + ""); + out.push('' + esc(n.label) + ""); out.push(""); }); out.push(""); @@ -1167,7 +2339,8 @@ go(); if ("ResizeObserver" in window) new ResizeObserver(go).observe(stage); else window.addEventListener("resize", go); - attachTips(stage); + attachTips(stage, { what: "a protein or a line for its numbers." }); + roving(stage); } function radius(aa, longest) { @@ -1199,7 +2372,7 @@ function proteinTip(n, genome) { return "" + esc(n.id) + "" + (n.gene ? " (" + esc(n.gene) + ")" : "") + (n.product ? ", " + esc(n.product.charAt(0).toLowerCase() + n.product.slice(1)) : "") + - "
" + fmt(n.aa) + " residues" + (genome ? ", " + esc(genome) : ""); + "
" + fmt(n.aa) + " residues" + (genome ? ", " + speciesHtml(genome) : ""); } function mountDupFamily(el) { @@ -1284,7 +2457,7 @@ var other = sides[1 - k]; var stage = panel.querySelector("[data-fig-stage]"); var head = panel.querySelector("[data-net-head]"); - if (head) head.innerHTML = "" + esc(side.label) + "" + esc(side.community) + ", " + side.nodes.length + " proteins, " + side.edges.length + " links"; + if (head) head.innerHTML = "" + speciesHtml(side.label) + "" + esc(side.community) + ", " + side.nodes.length + " proteins, " + side.edges.length + " links"; var byTwin = {}; side.nodes.forEach(function (n, i) { byTwin[n.twin] = i; @@ -1293,7 +2466,7 @@ return { x: n.x, y: n.y, r: radius(n.aa, longest), label: n.id, cls: "in", tip: proteinTip(n, side.label) }; }); var links = side.edges.map(function (e) { - return { a: e[0], b: e[1], cls: "kept", label: "", tip: "" + esc(side.nodes[e[0]].id) + " and " + esc(side.nodes[e[1]].id) + "
A link in " + esc(side.label) + "." }; + return { a: e[0], b: e[1], cls: "kept", label: "", tip: "" + esc(side.nodes[e[0]].id) + " and " + esc(side.nodes[e[1]].id) + "
A link in " + speciesHtml(side.label) + "." }; }); other.edges.forEach(function (e) { var key = keyOf(other, e); @@ -1301,7 +2474,7 @@ var a = byTwin[other.nodes[e[0]].twin]; var b = byTwin[other.nodes[e[1]].twin]; if (a == null || b == null) return; - links.unshift({ a: a, b: b, cls: "ghost", label: "", tip: "" + esc(side.nodes[a].id) + " and " + esc(side.nodes[b].id) + "
A link in " + esc(other.label) + ", not in " + esc(side.label) + "." }); + links.unshift({ a: a, b: b, cls: "ghost", label: "", tip: "" + esc(side.nodes[a].id) + " and " + esc(side.nodes[b].id) + "
A link in " + speciesHtml(other.label) + ", not in " + speciesHtml(side.label) + "." }); }); var hull = hullOf(nodes); onWidth(stage, function (w) { @@ -1341,5 +2514,9 @@ document.querySelectorAll('[data-dup="family"]').forEach(mountDupFamily); document.querySelectorAll('[data-dup="communities"]').forEach(mountDupCommunities); - window.AMAPFigures = { draw: draw, render: render, tickLabel: tickLabel, attachTips: attachTips, rowsFor: rowsFor, windowSpec: windowSpec, fmt: fmt, esc: esc, stations: STATIONS, metroOrder: METRO_ORDER }; + window.AMAPFigures = { + draw: draw, render: render, tickLabel: tickLabel, attachTips: attachTips, rowsFor: rowsFor, windowSpec: windowSpec, fmt: fmt, esc: esc, tie: tie, + legend: legend, refocus: refocus, isShown: isShown, nameSubjects: nameSubjects, overlap: overlap, + stations: STATIONS, metroOrder: METRO_ORDER + }; })(); diff --git a/docs/assets/how.js b/docs/assets/how.js index ae7ea5f..7f62635 100644 --- a/docs/assets/how.js +++ b/docs/assets/how.js @@ -36,7 +36,7 @@ var STORIES = [ { - key: "agree", group: "The four cases", label: "Agrees", tag: "Case 2", + key: "agree", group: "The five outcomes, in eight stories", label: "Agrees", tag: "Case 2", panels: [{ key: "case2", from: 1960, to: 3300, tick: 500 }], text: [ "Bakta calls dnaN from 2,052 to 3,260 on the plus strand.", @@ -52,7 +52,23 @@ ] }, { - key: "late-start", group: "The four cases", label: "Starts late", tag: "Case 4", + key: "same-ends", group: "The five outcomes, in eight stories", label: "Same ends, other length", tag: "Case 3", + panels: [{ key: "case3", from: 23700, to: 25600, tick: 500 }], + text: [ + "Bakta calls fhaA from 23,861 to 25,426 on the minus strand: 1,566 bp.", + "H37Rv's Rv0020c, fhaA, aligns over exactly the same stretch, the whole protein to its stop codon. Its protein is six residues longer: miniprot aligns 1,584 nt of it on these 1,566 bp, with a gap where CDC1551 lacks them.", + "Same start, same end, same strand, but not the same length: 1,566 bp called, 1,584 nt aligned. That is case 3, an imperfect match.", + "Nothing to settle. A case 3 call is reliable enough: it stays as Bakta made it, tied to Rv0020c.", + "It ends in a stop codon: nothing to change.", + NOTHING[6], + "It keeps Bakta's locus tag, name and product, with Rv0020c in its Dbxref." + ], + out: [ + "L4_CDC1551 AMAP CDS 23861 25426 . - 0 ID=L4_CDC1551_00025;Parent=gene-L4_CDC1551_00025;Name=fhaA;product=Conserved protein with FHA domain%2C FhaA;Dbxref=AMAP_ids:Rv0020c" + ] + }, + { + key: "late-start", group: "The five outcomes, in eight stories", label: "Starts late", tag: "Case 4", panels: [{ key: "case4-start", from: -60, to: 1640, tick: 500, marks: [{ s: 1, e: 33, rows: [0, 1], label: "33 bp", labelAt: "right", from: 3 }] }], text: [ "Bakta calls dnaA from base 34 to 1,524.", @@ -66,7 +82,7 @@ out: ["L4_CDC1551 AMAP CDS 1 1524 . + 0 ID=L4_CDC1551_Rv0001;Parent=gene-L4_CDC1551_Rv0001;Name=;product=;Dbxref=AMAP_ids:Rv0001"] }, { - key: "split", group: "The four cases", label: "One gene, two calls", tag: "Case 4", + key: "split", group: "The five outcomes, in eight stories", label: "One gene, two calls", tag: "Case 4", panels: [{ key: "case4-split", from: 9850, to: 10950, tick: 250 }], text: [ "Bakta calls two short genes, from 9,963 to 10,160 and from 10,523 to 10,828, both hypothetical proteins.", @@ -80,7 +96,7 @@ out: ["L4_CDC1551 AMAP CDS 9914 10828 . + 0 ID=L4_CDC1551_Rv0007;Parent=gene-L4_CDC1551_Rv0007;Name=;product=;Dbxref=AMAP_ids:Rv0007"] }, { - key: "longer", group: "The four cases", label: "Much longer", tag: "Case 4", + key: "longer", group: "The five outcomes, in eight stories", label: "Much longer", tag: "Case 4", panels: [{ key: "case4-longer", from: 2267500, to: 2272800, tick: 1000, marks: [{ s: 2267753, e: 2271055, rows: [0, 1], label: "3,303 bp", from: 3 }] }], text: [ "Bakta calls a 4,821 bp site-specific DNA methyltransferase on the minus strand.", @@ -94,7 +110,7 @@ out: ["L4_CDC1551 AMAP CDS 2267753 2272573 . - 0 ID=L4_CDC1551_02139;Parent=gene-L4_CDC1551_02139;Name=;product=site-specific DNA-methyltransferase (adenine-specific);Dbxref=AMAP_ids:Rv2024c"] }, { - key: "new", group: "The four cases", label: "Not in the reference", tag: "Case 1", + key: "new", group: "The five outcomes, in eight stories", label: "Not in the reference", tag: "Case 1", panels: [{ key: "case1-kept", from: 1979300, to: 1985800, tick: 2000 }], text: [ "Bakta calls four genes in a row: a mannosyltransferase, a sulfite oxidase-like oxidoreductase, an MMPL transporter and a hypothetical protein.", @@ -111,7 +127,7 @@ ] }, { - key: "false", group: "The four cases", label: "A known false call", tag: "Case 1", + key: "false", group: "The five outcomes, in eight stories", label: "A known false call", tag: "Case 1", panels: [{ key: "case1-dropped", from: 11330, to: 11620, tick: 100 }], text: [ "Bakta calls a 108 bp hypothetical protein on the minus strand.", @@ -125,7 +141,7 @@ out: [] }, { - key: "missed", group: "The four cases", label: "Missed by Bakta", tag: "Missing", + key: "missed", group: "The five outcomes, in eight stories", label: "Missed by Bakta", tag: "Missing", panels: [{ key: "missing", from: 33100, to: 34420, tick: 250, marks: [{ s: 33526, e: 33738, rows: [0, 1], label: "no call", kind: "ref", from: 3 }] }], text: [ "Bakta calls a gene from 33,168 to 33,497 and bioF2 from 34,239, and nothing between them.", @@ -175,8 +191,10 @@ { key: "contig-end", group: "The harder ones", label: "Cut by a contig end", tag: "Partial", panels: [ - { key: "contig-end", from: 100250, to: 102146, tick: 500, title: "The end of contig_10", edges: [{ at: 102146.5, label: "end of contig_10", side: "left", from: 1 }] }, - { key: "contig-end-2", from: 3700, to: 4591, tick: 250, title: "The end of contig_9", edges: [{ at: 4591.5, label: "end of contig_9", side: "left", from: 1 }] } + { key: "contig-end", from: 100250, to: 102146, tick: 500, title: "The end of contig_10", edges: [{ at: 102146.5, label: "end of contig_10", side: "left", from: 1 }], + marks: [{ s: 102120, e: 102146, rows: [3, 3], label: "partial: 3' end", labelAt: "left", from: 5 }] }, + { key: "contig-end-2", from: 3700, to: 4591, tick: 250, title: "The end of contig_9", edges: [{ at: 4591.5, label: "end of contig_9", side: "left", from: 1 }], + marks: [{ s: 4565, e: 4591, rows: [3, 3], label: "partial: 5' end", labelAt: "left", from: 5 }] } ], text: [ "CDC1551 cut into 40 contigs. Near the end of contig_10 Bakta calls Rv1240's gene and two short genes on the minus strand, and nothing on the plus strand. At the end of contig_9 it calls sugC and a short hypothetical protein.", @@ -247,6 +265,15 @@ return Math.max(0, Math.min(a.e, b.e) - Math.max(a.s, b.s) + 1); } + function copyOf(g, name) { + var c = {}; + Object.keys(g).forEach(function (k) { + c[k] = g[k]; + }); + c.name = name; + return c; + } + function panelSpec(story, panel) { var parts = F.rowsFor(panel.key); var win = parts.win; @@ -273,14 +300,46 @@ } } if (parts.bad.length) rows.push({ label: "Known false\npositives", cls: "drop", genes: parts.bad, from: 2 }); - var amap = parts.amap.map(function (g, i) { - if (story.labelIds) g.name = win.amap[i].id.replace(win.base + "_", ""); - return g; - }); - var changed = !sameGenes(parts.corrected, amap); + // With labelIds the genes go by their IDs. The copy's _2 comes from the + // step that sorts out copies, so until then it goes by the protein's + // name alone. + var idOf = function (id) { + return id.replace(win.base + "_", ""); + }; + var corrected = story.labelIds ? parts.corrected.map(function (g) { + return copyOf(g, idOf(g.id)); + }) : parts.corrected; + var final = parts.amap; + var renamed = null; + if (story.labelIds) { + renamed = parts.amap.map(function (g, i) { + return copyOf(g, idOf(win.amap[i].id)); + }); + final = renamed.map(function (g, i) { + return copyOf(g, win.amap[i].rvs ? g.name.replace(/_\d+$/, "") : g.name); + }); + if (renamed.map(function (g) { + return g.name; + }).join("|") === final.map(function (g) { + return g.name; + }).join("|")) renamed = null; + } + var changed = !sameGenes(corrected, final); var amapRow = rows.length; - rows.push({ label: "AMAP", cls: "amap", genes: parts.corrected, empty: parts.corrected.length ? null : "nothing kept", from: 4, until: changed ? 4 : null }); - if (changed) rows.push({ label: "AMAP", cls: "amap", genes: amap, overlay: true, from: 5 }); + var layers = [corrected]; + if (changed) layers.push(final); + if (renamed) layers.push(renamed); + // Each layer of what AMAP keeps from its own step: the classification's + // genes at 4, the translated ones at 5, the copies named at 6. + var steps = [4, changed ? 5 : null, renamed ? 6 : null].filter(function (n) { + return n; + }); + layers.forEach(function (genes, i) { + rows.push({ + label: "AMAP", cls: "amap", genes: genes, overlay: i > 0, empty: i === 0 && !genes.length ? "nothing kept" : null, + from: steps[i], until: i + 1 < layers.length ? steps[i + 1] - 1 : null + }); + }); var ribbons = []; win.bakta.forEach(function (c, i) { win.ref.forEach(function (r, j) { @@ -301,13 +360,69 @@ }); return copy; }); - return { + return F.nameSubjects({ from: panel.from, to: panel.to, labelW: 96, geneH: 18, rowGap: 24, top: 20, alt: story.label + ": " + story.text[0], + refName: reference, phased: true, rows: rows, ribbons: ribbons, marks: marks, edges: panel.edges || [], axis: true, tick: panel.tick, tickLabel: F.tickLabel(panel.to - panel.from) + }); + } + + // Which steps change the drawing of a story, or the lines it ends on: the + // others go by quicker when it plays, and say so in the list of steps. + function changesOf(story) { + var ch = [false, true, false, false, false, false, false, false]; + var note = function (o) { + if (o.from) ch[o.from] = true; + if (o.until && o.until < 7) ch[o.until + 1] = true; }; + story.panels.forEach(function (panel) { + var spec = panelSpec(story, panel); + spec.rows.forEach(note); + spec.ribbons.forEach(note); + spec.marks.forEach(note); + spec.edges.forEach(note); + spec.rows.forEach(function (row) { + row.genes.forEach(function (g) { + // Calls with nothing aligned are dashed at 3, and those dropped + // fade at 4. + if (/c1-call/.test(g.phaseCls || "")) ch[3] = true; + if (/ph-dim/.test(g.phaseCls || "")) ch[4] = true; + }); + }); + }); + if (story.out.length) ch[7] = true; + return ch; + } + + // A step's text, with the species named in italics as everywhere else, + // and the words that stay together as the docs keep them (F.tie). + function setText(el, text) { + el.textContent = ""; + String(text).split(/\b([A-Z]\. (?:tuberculosis|bovis|coli))\b/).forEach(function (part, i) { + if (!part) return; + if (i % 2) { + var em = document.createElement("i"); + em.textContent = part.replace(" ", "\u00a0"); + el.appendChild(em); + } else { + el.appendChild(document.createTextNode(F.tie(part))); + } + }); + } + + function words(text) { + return String(text).split(/\s+/).filter(Boolean).length; + } + + // How long Play stays on a step: time to read it, between 3 and 22 s, and + // 2 s for a step that changes nothing. + function dwell(story, n, changes) { + var ch = changes || changesOf(story); + if (!ch[n]) return 2000; + return Math.max(3000, Math.min(22000, 1500 + 300 * words(story.text[n - 1]))); } /* โ”€โ”€ The page โ”€โ”€ */ @@ -316,7 +431,9 @@ if (!root) return; var picker = root.querySelector("[data-stories]"); var stepList = root.querySelector("[data-steps]"); + var card = root.querySelector(".how-stage-card"); var stage = root.querySelector("[data-stage]"); + var textBox = root.querySelector(".how-text"); var kicker = root.querySelector("[data-kicker]"); var title = root.querySelector("[data-step-title]"); var text = root.querySelector("[data-step-text]"); @@ -325,60 +442,168 @@ var prev = root.querySelector("[data-prev]"); var next = root.querySelector("[data-next]"); var playBtn = root.querySelector("[data-play]"); + + // Each button holds both of its words and shows one, so it keeps the size + // of the longer and nothing beside or under it moves when the word changes. + function say(btn, words, second) { + if (!btn.querySelector(".say-2")) { + btn.innerHTML = '' + F.esc(words[0]) + '' + F.esc(words[1]) + ""; + } + btn.classList.toggle("said-2", !!second); + } var genomeLabel = root.querySelector("[data-genome]"); + var legendBox = root.querySelector(".how-legend"); - var story = STORIES[0]; + function find(key) { + return STORIES.filter(function (s) { + return s.key === key; + })[0]; + } + + // The story a page opens on, without a story in its address. + var story = find("agree") || STORIES[0]; var step = 1; var timer = null; - var redraws = []; + var changes = changesOf(story); - // The pickers, one row per group. - var groups = {}; + // The page carries the chips and the steps, so it does not move when this + // runs; a story the page lacks still gets a chip. STORIES.forEach(function (s) { - if (!groups[s.group]) { + if (picker.querySelector('[data-story="' + s.key + '"]')) return; + var group = null; + picker.querySelectorAll(".how-group").forEach(function (g) { + if (g.getAttribute("data-group") === s.group) group = g.querySelector(".seg"); + }); + if (!group) { var g = document.createElement("div"); g.className = "how-group"; - g.innerHTML = '
' + F.esc(s.group) + '
'; + g.setAttribute("data-group", s.group); + g.innerHTML = '

' + F.esc(s.group) + '

'; picker.appendChild(g); - groups[s.group] = g.querySelector(".seg"); + group = g.querySelector(".seg"); } - var b = document.createElement("button"); - b.type = "button"; - b.setAttribute("data-story", s.key); - b.setAttribute("aria-pressed", "false"); - b.innerHTML = F.esc(s.label) + ' ' + F.esc(s.tag) + ""; - groups[s.group].appendChild(b); + var a = document.createElement("a"); + a.href = "how-it-works.html#" + s.key; + a.setAttribute("data-story", s.key); + a.innerHTML = F.esc(s.label) + ' ' + F.esc(s.tag) + ""; + group.appendChild(a); }); + if (!stepList.querySelector("[data-step]")) { + STEPS.forEach(function (name, i) { + var li = document.createElement("li"); + li.innerHTML = '"; + stepList.appendChild(li); + }); + } - STEPS.forEach(function (name, i) { - var li = document.createElement("li"); - li.innerHTML = '"; - stepList.appendChild(li); - }); + function address() { + try { + history.replaceState(null, "", "#" + story.key + (step > 1 ? "/" + step : "")); + } catch (e) { + /* file:// in a strict browser */ + } + } - function showStep() { + function flash(el) { + if (reduceMotion) return; + el.classList.remove("just-on"); + void el.getBoundingClientRect(); + el.classList.add("just-on"); + setTimeout(function () { + el.classList.remove("just-on"); + }, 750); + } + + function showStep(moved) { stage.querySelectorAll("[data-from]").forEach(function (el) { var from = +el.getAttribute("data-from"); var until = +el.getAttribute("data-until") || 99; - el.classList.toggle("step-off", step < from || step > until); + var off = step < from || step > until; + var was = el.classList.contains("step-off"); + el.classList.toggle("step-off", off); + if (moved && was && !off) flash(el); + }); + stage.querySelectorAll("[data-ghost-from]").forEach(function (el) { + var ghost = step < +el.getAttribute("data-ghost-from"); + el.classList.toggle("ghost", ghost); + // A row still to come is named when it arrives, not before. + if (ghost) el.setAttribute("aria-hidden", "true"); + else el.removeAttribute("aria-hidden"); }); - stepList.querySelectorAll("button").forEach(function (b) { + // The calls with nothing aligned are dashed once compared (3), and the + // ones dropped fade once the disagreements are settled (4). + stage.querySelectorAll("svg").forEach(function (svg) { + svg.setAttribute("data-phase", String(Math.min(step, 4))); + F.refocus(svg); + }); + stepList.querySelectorAll("[data-step]").forEach(function (b) { var n = +b.getAttribute("data-step"); b.setAttribute("aria-current", n === step ? "step" : "false"); b.classList.toggle("done", n < step); }); kicker.textContent = "Step " + step + " of 7 ยท " + story.label; title.textContent = STEPS[step - 1]; - text.textContent = story.text[step - 1]; + setText(text, story.text[step - 1]); outBox.hidden = step < 7 || !story.out.length; - progress.style.width = (step / 7) * 100 + "%"; - prev.disabled = step === 1; - next.textContent = step === 7 ? "Start again" : "Next"; + if (!timer) { + progress.style.transition = ""; + progress.style.width = (step / 7) * 100 + "%"; + } + prev.setAttribute("aria-disabled", step === 1 ? "true" : "false"); + say(next, ["Next", "Start again"], step === 7); + } + + // The text under the drawing keeps the height of the story's longest step, + // so nothing below it moves while stepping. On a phone the step 7 file + // excerpt is left out of that height: it would open every story on a + // screenful of empty space, and the controls sit above the text, so what + // it pushes down is only the page under it. + // What a story writes, one GFF3 line after another: a line breaks between + // its attributes, after a ';', rather than inside a name. + function fillOut(pre, lines) { + pre.textContent = ""; + lines.forEach(function (line, k) { + if (k) pre.appendChild(document.createTextNode("\n")); + line.split(/(;)/).forEach(function (piece) { + if (!piece) return; + pre.appendChild(document.createTextNode(piece)); + if (piece === ";") pre.appendChild(document.createElement("wbr")); + }); + }); + } + + function fitText() { + var w = textBox.offsetWidth; + if (!w) return; + var probe = textBox.cloneNode(true); + probe.removeAttribute("aria-live"); + probe.setAttribute("aria-hidden", "true"); + probe.style.cssText = "position:absolute;visibility:hidden;pointer-events:none;left:0;top:0;min-height:0;width:" + w + "px"; + card.appendChild(probe); + var parts = ["[data-kicker]", "[data-step-title]", "[data-step-text]"].map(function (sel) { + return probe.querySelector(sel); + }); + var out = probe.querySelector("[data-out]"); + fillOut(out.querySelector("pre"), story.out); + var tallest = 0; + for (var n = 1; n <= 7; n++) { + parts[0].textContent = "Step " + n + " of 7 ยท " + story.label; + parts[1].textContent = STEPS[n - 1]; + setText(parts[2], story.text[n - 1]); + out.hidden = n < 7 || !story.out.length || w < 600; + tallest = Math.max(tallest, probe.offsetHeight); + } + probe.remove(); + textBox.style.minHeight = tallest + "px"; } - function drawStory() { + // center: bring the story's chip into the middle of its row, when the + // reader picked the story or a link named it; on a first visit the rows + // start at their first chip. + function drawStory(center) { + stop(); stage.innerHTML = ""; - redraws = []; + var used = {}; story.panels.forEach(function (panel) { var box = document.createElement("div"); box.className = "how-panel"; @@ -392,45 +617,103 @@ holder.className = "how-draw"; box.appendChild(holder); stage.appendChild(box); - redraws.push(F.render(holder, function () { + F.render(holder, function () { return panelSpec(story, panel); - }, showStep)); + }, function (svg, spec) { + Object.keys(spec._used || {}).forEach(function (k) { + used[k] = 1; + }); + showStep(); + }); F.attachTips(holder); }); var win = DATA.windows[story.panels[0].key]; - genomeLabel.textContent = win.genome + (/coli/.test(win.genome) ? ", against K-12" : ", against H37Rv"); - outBox.querySelector("pre").textContent = story.out.join("\n"); + // Species in italics, as everywhere else on the site. + genomeLabel.innerHTML = F.esc(win.genome).replace(/^([A-Z]\.) ([a-z]+)/, "$1 $2") + (/coli/.test(win.genome) ? ", against K-12" : ", against H37Rv"); + if (legendBox) legendBox.innerHTML = F.legend(used, /coli/.test(win.genome) ? "K-12" : "H37Rv"); + fillOut(outBox.querySelector("pre"), story.out); picker.querySelectorAll("[data-story]").forEach(function (b) { - b.setAttribute("aria-pressed", b.getAttribute("data-story") === story.key ? "true" : "false"); + if (b.getAttribute("data-story") === story.key) b.setAttribute("aria-current", "true"); + else b.removeAttribute("aria-current"); + }); + // Steps that change nothing in this story show hollow. + changes = changesOf(story); + stepList.querySelectorAll("[data-step]").forEach(function (b) { + var n = +b.getAttribute("data-step"); + var same = !changes[n]; + b.classList.toggle("same", same); + var note = b.querySelector(".how-same"); + if (note) note.remove(); + // A hollow step is named with its note, as one phrase. + if (same) b.setAttribute("aria-label", n + " " + STEPS[n - 1] + ", no change in this story"); + else b.removeAttribute("aria-label"); + if (same) b.title = "No change in this story"; + else b.removeAttribute("title"); }); showStep(); + fitText(); + // The picked story's chip in view, in a row that scrolls sideways. + var chip = center && picker.querySelector('[data-story="' + story.key + '"]'); + var row = chip && chip.parentNode; + if (row && row.scrollWidth > row.clientWidth + 2) { + row.scrollLeft = Math.max(0, chip.offsetLeft - row.offsetLeft - (row.clientWidth - chip.offsetWidth) / 2); + } + } + + // The drawing in view when a story is picked or opened by a link, when it + // is not already. + function reveal() { + var r = card.getBoundingClientRect(); + if (r.top < 64 || r.top + Math.min(r.height, 320) > window.innerHeight) { + card.scrollIntoView({ block: "start", behavior: reduceMotion ? "auto" : "smooth" }); + } } function stop() { clearTimeout(timer); timer = null; - playBtn.textContent = "Play all"; + say(playBtn, ["Play", "Pause"], false); + card.classList.remove("playing"); + progress.style.transition = ""; + progress.style.width = (step / 7) * 100 + "%"; } function go(n) { + var before = step; step = Math.max(1, Math.min(7, n)); - showStep(); + showStep(step !== before); + address(); + } + + // While it plays, the bar runs on to the next step in the time the step + // is given. + function tick() { + if (step >= 7) { + stop(); + return; + } + var ms = dwell(story, step, changes); + progress.style.transition = "none"; + progress.style.width = (step / 7) * 100 + "%"; + void progress.offsetWidth; + progress.style.transition = "width " + ms + "ms linear"; + progress.style.width = ((step + 1) / 7) * 100 + "%"; + timer = setTimeout(function () { + go(step + 1); + tick(); + }, ms); } picker.addEventListener("click", function (event) { var b = event.target.closest("[data-story]"); - if (!b) return; + if (!b || event.button || event.metaKey || event.ctrlKey || event.shiftKey || event.altKey) return; + event.preventDefault(); stop(); - story = STORIES.filter(function (s) { - return s.key === b.getAttribute("data-story"); - })[0]; + story = find(b.getAttribute("data-story")) || story; step = 1; - drawStory(); - try { - history.replaceState(null, "", "#" + story.key); - } catch (e) { - /* file:// in a strict browser */ - } + drawStory(true); + address(); + reveal(); }); stepList.addEventListener("click", function (event) { var b = event.target.closest("[data-step]"); @@ -439,6 +722,7 @@ go(+b.getAttribute("data-step")); }); prev.addEventListener("click", function () { + if (step === 1) return; stop(); go(step - 1); }); @@ -451,21 +735,15 @@ stop(); return; } - playBtn.textContent = "Pause"; if (step === 7) go(1); - (function tick() { - timer = setTimeout(function () { - if (step >= 7) { - stop(); - return; - } - go(step + 1); - tick(); - }, reduceMotion ? 3500 : 3000); - })(); + say(playBtn, ["Play", "Pause"], true); + card.classList.add("playing"); + tick(); }); root.addEventListener("keydown", function (event) { - if (/input|textarea|select/i.test(event.target.tagName)) return; + if (/input|textarea|select/i.test(event.target.tagName) || event.altKey || event.ctrlKey || event.metaKey) return; + // Inside a drawing the arrows move from gene to gene. + if (event.target.closest && event.target.closest("svg")) return; if (event.key === "ArrowRight") { stop(); go(step + 1); @@ -474,26 +752,53 @@ go(step - 1); } }); + // Which keys do what, while the keyboard is in the walk-through. + root.addEventListener("focusin", function (event) { + root.setAttribute("data-keys", event.target.closest && event.target.closest("svg") ? "genes" : "steps"); + }); + root.addEventListener("focusout", function (event) { + if (!event.relatedTarget || !root.contains(event.relatedTarget)) root.removeAttribute("data-keys"); + }); + if ("ResizeObserver" in window) { + var fitted = 0; + new ResizeObserver(function () { + if (Math.abs(textBox.offsetWidth - fitted) < 2) return; + fitted = textBox.offsetWidth; + fitText(); + }).observe(textBox); + } - // A story named in the address opens first, so a link can point at one. - function fromHash() { - var wanted = (location.hash || "").slice(1); - var found = STORIES.filter(function (s) { - return s.key === wanted; - })[0]; - if (found && found !== story) { - stop(); - story = found; - step = 1; - return true; - } - return false; + // A story named in the address opens first, so a link can point at one, + // and #key/4 at its fourth step. + function wanted() { + var m = /^#([\w-]+)(?:\/([1-7]))?$/.exec(location.hash || ""); + var found = m && find(m[1]); + return found ? { story: found, step: +(m[2] || 1) } : null; + } + var want = wanted(); + if (want) { + story = want.story; + step = want.step; } - fromHash(); - drawStory(); + drawStory(!!want); + if (want) reveal(); window.addEventListener("hashchange", function () { - if (fromHash()) drawStory(); + var w = wanted(); + if (!w || (w.story === story && w.step === step)) return; + stop(); + var redraw = w.story !== story; + story = w.story; + step = w.step; + if (redraw) drawStory(true); + else showStep(true); + reveal(); }); - window.AMAPHow = { stories: STORIES, rules: RULES }; + window.AMAPHow = { + stories: STORIES, rules: RULES, + dwell: function (key, n) { + var s = find(key); + return s ? dwell(s, n) : null; + } + }; })(); diff --git a/docs/assets/readme/hero-dark.gif b/docs/assets/readme/hero-dark.gif index 674a08d..5304a46 100644 Binary files a/docs/assets/readme/hero-dark.gif and b/docs/assets/readme/hero-dark.gif differ diff --git a/docs/assets/readme/hero-light.gif b/docs/assets/readme/hero-light.gif index 8fd82bf..10b724a 100644 Binary files a/docs/assets/readme/hero-light.gif and b/docs/assets/readme/hero-light.gif differ diff --git a/docs/assets/readme/metro-dark.png b/docs/assets/readme/metro-dark.png index 9feba4e..70dceb7 100644 Binary files a/docs/assets/readme/metro-dark.png and b/docs/assets/readme/metro-dark.png differ diff --git a/docs/assets/readme/metro-light.png b/docs/assets/readme/metro-light.png index bf1c44d..2fbd49b 100644 Binary files a/docs/assets/readme/metro-light.png and b/docs/assets/readme/metro-light.png differ diff --git a/docs/assets/readme/outcomes-dark.png b/docs/assets/readme/outcomes-dark.png index 90d831b..41ea2eb 100644 Binary files a/docs/assets/readme/outcomes-dark.png and b/docs/assets/readme/outcomes-dark.png differ diff --git a/docs/assets/readme/outcomes-light.png b/docs/assets/readme/outcomes-light.png index b8cac85..994c99a 100644 Binary files a/docs/assets/readme/outcomes-light.png and b/docs/assets/readme/outcomes-light.png differ diff --git a/docs/assets/report/amap-annotation-results.webp b/docs/assets/report/amap-annotation-results.webp index 54a9003..6dc5c0f 100644 Binary files a/docs/assets/report/amap-annotation-results.webp and b/docs/assets/report/amap-annotation-results.webp differ diff --git a/docs/assets/report/amap-cds-stats.webp b/docs/assets/report/amap-cds-stats.webp index 8b18157..27dc49f 100644 Binary files a/docs/assets/report/amap-cds-stats.webp and b/docs/assets/report/amap-cds-stats.webp differ diff --git a/docs/assets/report/master-table-explorer.webp b/docs/assets/report/master-table-explorer.webp index c3ed030..666f319 100644 Binary files a/docs/assets/report/master-table-explorer.webp and b/docs/assets/report/master-table-explorer.webp differ diff --git a/docs/assets/report/overview.webp b/docs/assets/report/overview.webp index 4f60f11..5e7ae9e 100644 Binary files a/docs/assets/report/overview.webp and b/docs/assets/report/overview.webp differ diff --git a/docs/assets/report/poster.webp b/docs/assets/report/poster.webp new file mode 100644 index 0000000..2e8cfa5 Binary files /dev/null and b/docs/assets/report/poster.webp differ diff --git a/docs/assets/search-index.js b/docs/assets/search-index.js index a44967e..cb7f7b2 100644 --- a/docs/assets/search-index.js +++ b/docs/assets/search-index.js @@ -1,1239 +1,10 @@ /* Generated by tools/build_docs.py: the documentation, section by section, for the search box. */ -window.AMAP_DOCS_INDEX = [ -{ -"page": "Tutorial", -"section": "", -"url": "getting-started/tutorial.html", -"text": "Every command spelled out, nothing assumed. It uses a real genome you can download in one line, so you can follow along and compare your numbers against the ones printed here. Those numbers come from actual runs of these commands, not from memory, made on 18 October 2026 with AMAP at commit 947a99d. The runs skipped eggNOG, with --skip_functional_annotation. None of the numbers depends on it, and the one table shown here that is written after it was made by hand from the same run, as the sample check explains. The genome is Mycobacterium tuberculosis CDC1551, corrected against the H37Rv refere" -}, -{ -"page": "Tutorial", -"section": "1. What you need", -"url": "getting-started/tutorial.html#1-what-you-need", -"text": "Nextflow and a container engine. Every tool the pipeline itself calls lives inside the AMAP image, so there is nothing else to install. The easiest way to get Nextflow is Conda. If you do not have it, Miniforge is the one to install: conda create -n nextflow-env -c conda-forge -c bioconda nextflow conda activate nextflow-env Nextflow needs Java 17 or newer. Check what you have: java -version nextflow -version Java 8 is not enough Nextflow will refuse to start on Java 8 with a message about an unsupported class file version. Install a newer JDK, for example conda install -c conda-forge openjdk=" -}, -{ -"page": "Tutorial", -"section": "The machine", -"url": "getting-started/tutorial.html#the-machine", -"text": "Worth checking before you start, because this is what stops a run rather than anything you will have got wrong. Disk goes first. The container is 12 GB unpacked. The eggNOG database is built on first use and cached: AMAP skips the 60 GB Diamond core that eggNOG-mapper would otherwise fetch and builds a bacteria-only index of about 3 GB instead, on top of the metadata it downloads. Nextflow keeps every intermediate under work/, which for one genome is a few GB and grows with the sample sheet. Memory decides whether the run finishes at all. Two steps are heavy: eggNOG annotation 24 50 GB eggNOG " -}, -{ -"page": "Tutorial", -"section": "2. Check the plumbing before touching real data", -"url": "getting-started/tutorial.html#2-check-the-plumbing-before-touching-real-data", -"text": "The test profile annotates the reference genome that ships inside the container, so it needs no data of your own: nextflow run PathoGenOmics-Lab/AMAP -profile docker,test The first run pulls the AMAP image. It is around 3.4 GB compressed and 12 GB on disk, because it carries the Bakta database and the H37Rv reference files. That is a one-off; later runs reuse it. If this finishes without an error, the whole chain works." -}, -{ -"page": "Tutorial", -"section": "3. Get a genome", -"url": "getting-started/tutorial.html#3-get-a-genome", -"text": "Make a working directory and fetch an assembly: mkdir amap-tutorial && cd amap-tutorial NCBI=https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF ASM=000/008/585/GCF_000008585.1_ASM858v1/GCF_000008585.1_ASM858v1_genomic curl -sL -o cdc1551.fna.gz \"$NCBI/$ASM.fna.gz\" gunzip cdc1551.fna.gz Have a look at what you got: grep '^>' cdc1551.fna >NC_002755.2 Mycobacterium tuberculosis CDC1551, complete sequence One closed contig, 4,403,837 bp. AMAP takes assembled genomes, so reads are not an input here." -}, -{ -"page": "Tutorial", -"section": "4. Write the sample sheet", -"url": "getting-started/tutorial.html#4-write-the-sample-sheet", -"text": "A tab-separated file with exactly two columns: printf 'sampleId\\tfasta\\nCDC1551\\t%s/cdc1551.fna\\n' \"$PWD\" > samples.tsv cat samples.tsv sampleId\tfasta CDC1551\t/path/to/amap-tutorial/cdc1551.fna It has to be a real tab Spaces will not do, and this is the single most common reason a sample sheet is rejected. Check with cat -t samples.tsv: real tabs show up as ^I. (cat -A does the same on Linux but is not accepted by the cat that ships with macOS.) Keep sample IDs to letters, digits and underscores. Several scripts cut the name at the first dot, so G26689.1 becomes G26689." -}, -{ -"page": "Tutorial", -"section": "5. Validate before committing to a long run", -"url": "getting-started/tutorial.html#5-validate-before-committing-to-a-long-run", -"text": "nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv --validate_only The checks write their verdict to results/pipeline_info/validation.log, which ends: tail -n 8 results/pipeline_info/validation.log --- FASTA Validation (1 samples) --- ============================================================ Results: 1 OK, 0 warnings, 0 failed Total: 0 error(s), 0 warning(s) ============================================================ โœ“ PASSED โ€” all 1 samples look good This costs seconds and catches the mistakes that would otherwise surface minutes in. Preparing inputs lists eve" -}, -{ -"page": "Tutorial", -"section": "6. Run it", -"url": "getting-started/tutorial.html#6-run-it", -"text": "nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv What happens, in order: lineage typing, Bakta annotation, alignment of the H37Rv proteome onto your genome, case classification, correction, eggNOG functional annotation, duplication analysis, pangenome, report. The first run also downloads and builds the eggNOG database, which is several gigabytes and dominates the wall clock. Point --eggnog_db at somewhere persistent so the next run reuses it: nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv --eggnog_db ~/databases/eggnog If the run dies partway," -}, -{ -"page": "Tutorial", -"section": "7. Look at what came out", -"url": "getting-started/tutorial.html#7-look-at-what-came-out", -"text": "ls -F results/ AMAP/ BAKTA/ CDS_CASES/ DUPLICATIONS/ PANGENE/ PATHOTYPR/ amap_report.html final_report.html pipeline_info/ A run without eggNOG, like the ones behind this page, has no DUPLICATIONS/, PANGENE/ or final_report.html. Open results/final_report.html first. It is self-contained, so it travels by email. results/amap_report.html goes through the same run question by question, and a run without eggNOG still gets it." -}, -{ -"page": "Tutorial", -"section": "Did the correction do anything?", -"url": "getting-started/tutorial.html#did-the-correction-do-anything", -"text": "column -t -s$'\\t' results/CDS_CASES/summary/*-summary.tsv Type Number case2 2813 coordinates matched the reference exactly case3 24 coordinates matched, alignment length differed case4 1100 overlapped the reference but disagreed, resolved case1_initial 141 Bakta called it, the reference has nothing t Read that as: of 4077 Bakta calls, 2813 already agreed with the reference, 1100 had to be adjudicated, and 97 reference genes Bakta had missed were rescued. Only 11 of the 141 apparently novel genes survived scrutiny, which is what you would" -}, -{ -"page": "Tutorial", -"section": "Sanity check the sample", -"url": "getting-started/tutorial.html#sanity-check-the-sample", -"text": "column -t -s$'\\t' results/AMAP//-AMAP.qc.tsv Six of its eleven columns: genome_size n_contigs gc_pct total_cds pseudogene_count coding_density_pct 4403837 1 65.61 4014 123 91.3 65.6% GC and 91.3% coding density are what a healthy MTBC genome looks like. A number well outside that is worth investigating before trusting the rest. This table is written after eggNOG, so a run without it has none. The line above was made by running that step's own script, GET_OUTPUTS, on the corrected genes of the run behind this page, with an eggNOG table that has no rows. " -}, -{ -"page": "Tutorial", -"section": "The files you will actually use", -"url": "getting-started/tutorial.html#the-files-you-will-actually-use", -"text": "Lineage typing puts the lineage in front of the sample ID, so CDC1551 comes out as L4_CDC1551: Corrected annotation results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gff3 Same, GenBank results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gbk Proteins for downstream tools results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.clean.faa Everything known per gene results/AMAP/L4_CDC1551/L4_CDC1551-AMAPmastertable.tsv Tool versions for your methods results/pipeline_info/software_versions.yml Two protein files, and they differ on purpose *-AMAP.faa has one entry per annotated gene, pseudogenes included, so it matches the GFF3 gene for " -}, -{ -"page": "Tutorial", -"section": "8. More than one genome", -"url": "getting-started/tutorial.html#8-more-than-one-genome", -"text": "Add rows. The pipeline parallelises across samples, and the pangenome and the cross-genome duplication analysis only become meaningful with several: sampleId\tfasta CDC1551\t/data/cdc1551.fna Erdman\t/data/erdman.fna Beijing_1\t/data/beijing_1.fna Past a couple of dozen, move to a cluster: nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \\ --tsv samples.tsv --eggnog_db /shared/databases/eggnog Deployment on HPC covers scratch, shared filesystems and where things should live." -}, -{ -"page": "Tutorial", -"section": "9. The same thing for another organism", -"url": "getting-started/tutorial.html#9-the-same-thing-for-another-organism", -"text": "Everything above assumed MTBC. For anything else you supply the trusted reference yourself, and turn off the two MTBC-specific steps. Fetch a reference and a sample. Here, E. coli O157:H7 Sakai corrected against K-12 MG1655: NCBI=https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF K12=000/005/845/GCF_000005845.2_ASM584v2/GCF_000005845.2_ASM584v2_genomic SAKAI=000/008/865/GCF_000008865.2_ASM886v2/GCF_000008865.2_ASM886v2_genomic # the reference: annotation plus the genome it describes curl -sL -o ref.gbk.gz \"$NCBI/$K12.gbff. nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv ecoli.tsv " -}, -{ -"page": "Tutorial", -"section": "When something goes wrong", -"url": "getting-started/tutorial.html#when-something-goes-wrong", -"text": "Nextflow prints the work directory of the failed task. Everything you need is in it: work/ab/cd1234.../ โ”œโ”€โ”€ .command.sh the exact script that ran โ”œโ”€โ”€ .command.err stderr on its own, almost always the answer โ””โ”€โ”€ .exitcode Troubleshooting is organised by the error you are looking at." -}, -{ -"page": "Tutorial", -"section": "Where to go next", -"url": "getting-started/tutorial.html#where-to-go-next", -"text": "Parameters: every flag and its default How it works: what the correction actually does, and the thresholds behind it Outputs: every file the run produces" -}, -{ -"page": "Installation", -"section": "", -"url": "getting-started/installation.html", -"text": "AMAP needs two things on your machine: Nextflow, which runs the pipeline, and a container engine, which supplies every tool the pipeline calls. You do not install Bakta, Miniprot, eggNOG-mapper or any of the rest yourself; they all live inside the AMAP image. You also do not need to clone this repository. Nextflow pulls the pipeline straight from GitHub." -}, -{ -"page": "Installation", -"section": "1. Nextflow", -"url": "getting-started/installation.html#1-nextflow", -"text": "Nextflow needs Java 17 or newer, and Conda brings both: conda create -n nextflow-env conda activate nextflow-env conda install -c bioconda nextflow If you do not have Conda, Miniconda is a lightweight installer. Any other Nextflow installation works equally well, see the Nextflow documentation. AMAP requires Nextflow 24.04 or newer, which is enforced by the pipeline manifest." -}, -{ -"page": "Installation", -"section": "2. A container engine", -"url": "getting-started/installation.html#2-a-container-engine", -"text": "Pick whichever your machine allows. Docker Best on a laptop or workstation where you have root. Follow the Docker installation guide for your platform. Singularity / Apptainer The usual choice on an HPC cluster, where Docker is normally not permitted. Follow the Apptainer installation guide. Many clusters already provide it as a module: module load singularity # or: module load apptainer" -}, -{ -"page": "Installation", -"section": "3. Check the installation", -"url": "getting-started/installation.html#3-check-the-installation", -"text": "nextflow -version docker --version # or: singularity --version" -}, -{ -"page": "Installation", -"section": "4. Run the test profile", -"url": "getting-started/installation.html#4-run-the-test-profile", -"text": "Before pointing AMAP at real data, confirm the whole chain works. The test profile annotates the reference genome bundled in the container, so it needs no data of your own and nothing to download beyond the image: nextflow run PathoGenOmics-Lab/AMAP -profile docker,test The first run downloads the AMAP image, which is several gigabytes because it bundles the Bakta database and the H37Rv reference files. Expect that to take a while. Later runs reuse the cached image. Check inputs before a long run Once you have your own samples, --validate_only checks every FASTA and database path without execu" -}, -{ -"page": "Installation", -"section": "Running without a container", -"url": "getting-started/installation.html#running-without-a-container", -"text": "Possible but not recommended: you take on installing Bakta, Miniprot, eggNOG-mapper, Diamond, pangene, pathotypr, R and the report packages yourself, and pointing AMAP at each reference file with the reference arguments. The repository ships an environment.yml that reproduces the software stack under Conda: conda env create -f environment.yml conda activate amap You will still need to supply the Bakta database and the reference files, which the container would otherwise provide. See containers for what the image contains." -}, -{ -"page": "Installation", -"section": "Next steps", -"url": "getting-started/installation.html#next-steps", -"text": "Quickstart: your first real run Preparing inputs: the sample sheet and what makes a genome pass validation" -}, -{ -"page": "Quickstart", -"section": "", -"url": "getting-started/quickstart.html", -"text": "This walks from an installed Nextflow to an open report. If Nextflow or a container engine is still missing, start at installation." -}, -{ -"page": "Quickstart", -"section": "1. Write a sample sheet", -"url": "getting-started/quickstart.html#1-write-a-sample-sheet", -"text": "AMAP reads a tab-separated file with two columns, sampleId and fasta: sampleId\tfasta G26689\t/data/genomes/G26689.fasta G26690\t/data/genomes/G26690.fasta The separator must be a real tab. Paths can be absolute, or relative to where you launch Nextflow. See preparing inputs for the details and the checks AMAP runs over them." -}, -{ -"page": "Quickstart", -"section": "2. Check the inputs", -"url": "getting-started/quickstart.html#2-check-the-inputs", -"text": "Cheap, and it catches the mistakes that would otherwise surface twenty minutes into a run: nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv --validate_only" -}, -{ -"page": "Quickstart", -"section": "3. Run it", -"url": "getting-started/quickstart.html#3-run-it", -"text": "Pick what you have, and copy the command: Or start from one of these: nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv Docker nextflow run PathoGenOmics-Lab/AMAP -profile singularity --tsv samples.tsv Singularity nextflow run PathoGenOmics-Lab/AMAP \\ -profile singularity,slurm --tsv samples.tsv SLURM cluster Without a scheduler profile every task is capped at the machine's CPUs and 16 GB, and with the scheduler profiles at 48 CPUs and 128 GB, whichever order the profiles come in; a larger request, such as the 50 GB the eggNOG annotation asks for, is lowered to the cap." -}, -{ -"page": "Quickstart", -"section": "4. Read the results", -"url": "getting-started/quickstart.html#4-read-the-results", -"text": "Everything lands under results/ unless you set --outdir. The three things to open first: The full interactive report results/final_report.html Corrected annotation, per sample results/AMAP/{sampleId}/*-AMAP.gff3 Protein sequences for downstream tools results/AMAP/{sampleId}/*-AMAP.clean.faa Which protein file do I want? *-AMAP.faa holds every gene, pseudogenes included, so it mirrors the annotation exactly. *-AMAP.clean.faa drops pseudogenes and sequences with internal stop codons, which is what orthology, phylogenomics and dN/dS tools expect. Reach for the clean one unless you specifically ne" -}, -{ -"page": "Quickstart", -"section": "5. Resume an interrupted run", -"url": "getting-started/quickstart.html#5-resume-an-interrupted-run", -"text": "Nextflow caches completed tasks. If a run dies, restart it with -resume and it picks up where it stopped: nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv -resume Keep the same parameters. Changing an input invalidates the cache for every task downstream of it, which is what you want, but it does mean the run redoes that work." -}, -{ -"page": "Quickstart", -"section": "Common variations", -"url": "getting-started/quickstart.html#common-variations", -"text": "A non-MTBC organism. Lineage typing and Mycobrowser are specific to tuberculosis, so turn them off and supply your own reference: nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv \\ --skip_typing --skip_mycobrowser \\ --genus Escherichia --species \"Escherichia coli\" \\ --ref_annotation my_reference.gbk --ref_genome my_reference.fasta See custom reference. Just the corrected genes. If you do not need functional annotation, duplications, the pangenome or the report, stop after the correction step. It is much faster, and leaves the corrected genes as a table wi" -}, -{ -"page": "Quickstart", -"section": "Next steps", -"url": "getting-started/quickstart.html#next-steps", -"text": "Running AMAP: profiles, schedulers and larger runs Parameters: the full reference How it works: what the correction actually does" -}, -{ -"page": "Preparing inputs", -"section": "", -"url": "getting-started/inputs.html", -"text": "AMAP takes assembled genomes. It does not do assembly, so reads are not an input here." -}, -{ -"page": "Preparing inputs", -"section": "The sample sheet", -"url": "getting-started/inputs.html#the-sample-sheet", -"text": "A tab-separated file with a header row and exactly two columns: sampleId\tfasta G26689\t/data/genomes/G26689.fasta G26690\t/data/genomes/G26690.fasta L4_G12345\t./assemblies/L4_G12345.fa sampleId: The name carried through every output directory and file. Must be unique. Lineage typing puts the lineage in front, so G26689 comes out as L4_G26689. fasta: Path to the assembled genome. Absolute, or relative to the directory you launch Nextflow from. The separator has to be an actual tab. Spaces will not do, and this is the most common reason a sample sheet is rejected. Avoid dots in sample IDs Several " -}, -{ -"page": "Preparing inputs", -"section": "Pre-flight validation", -"url": "getting-started/inputs.html#pre-flight-validation", -"text": "VALIDATE_INPUTS runs before any computation, on every run. To run only the checks and stop: nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv --validate_only Results are written to results/pipeline_info/validation.log." -}, -{ -"page": "Preparing inputs", -"section": "What fails the run", -"url": "getting-started/inputs.html#what-fails-the-run", -"text": "Genome size below 100 kb, or above 20 Mb N content above 10% GC content below 20% or above 80% Empty sequences any record with no bases Duplicate contig headers any repeated identifier Sample sheet missing sampleId or fasta column, no data rows, duplicate or empty sample IDs FASTA path file does not exist or cannot be parsed Bakta database not found, or no version.json or bakta.db in it eggNOG database eggnog_db.ok in it while the database is incomplete, since Nextflow then never builds it again Reference files --ref_annotation, --ref_genome or a reference argument that the run reads but that " -}, -{ -"page": "Preparing inputs", -"section": "What only warns", -"url": "getting-started/inputs.html#what-only-warns", -"text": "Genome size below 500 kb, or above 10 Mb N content above 1% GC content below 25% or above 75% Assembly fragmentation more than 100 contigs, and again above 500 Short contigs any contig under 200 bp in a multi-contig assembly Non-standard characters anything outside ATGCNRYSWKMBDHV eggNOG database a bacteria.dmnd under 100 MB, which EGGNOG_DB takes for a truncated download and builds again Mycobrowser table --mycobrowser_ref neither on the host nor inside the container, so the master table goes without Mycobrowser categories Warnings do not stop a run. Add --strict_validation to have them stop " -}, -{ -"page": "Preparing inputs", -"section": "Assembly quality", -"url": "getting-started/inputs.html#assembly-quality", -"text": "The correction compares your annotation against a reference proteome, so assembly quality propagates directly into the result. Contiguity matters. A gene broken across two contigs cannot be recovered. The more fragmented the assembly, the more reference genes turn up as missing rather than corrected. Ns break coding sequences. A run of Ns inside a gene stops it translating, so it is dropped or flagged as a pseudogene. The 1% warning is worth taking seriously. Contig names are kept AMAP runs Bakta with --keep-contig-headers, so the sequence names you supplied are the ones in BAKTA/ and in the c" -}, -{ -"page": "Preparing inputs", -"section": "Reference data", -"url": "getting-started/inputs.html#reference-data", -"text": "For MTBC you need nothing else. The container bundles the H37Rv proteome, the Bakta light database, the Mycobrowser table and the pathotypr lineage markers. See containers for the full list. For any other organism, supply your own reference with --ref_annotation and --ref_genome. See custom reference." -}, -{ -"page": "Preparing inputs", -"section": "Next steps", -"url": "getting-started/inputs.html#next-steps", -"text": "Quickstart: run it Parameters: the full flag reference" -}, -{ -"page": "Parameters", -"section": "", -"url": "user-guide/parameters.html", -"text": "Every AMAP parameter, what it defaults to, and when you would change it. Run nextflow run PathoGenOmics-Lab/AMAP --help for the same list at the terminal. Pipeline parameters take two dashes (--tsv). Nextflow's own options take one (-profile, -resume)." -}, -{ -"page": "Parameters", -"section": "Input and output", -"url": "user-guide/parameters.html#input-and-output", -"text": "--tsv Sample sheet with sampleId and fasta columns, tab separated. Not needed with -profile test. file samples.tsv Yes --outdir Directory for results. string results No" -}, -{ -"page": "Parameters", -"section": "Skip options", -"url": "user-guide/parameters.html#skip-options", -"text": "--skip_typing Skip pathotypr lineage typing. Required for non-MTBC organisms. boolean false --skip_functional_annotation Stop after annotation correction. Skips eggNOG, duplications, pangenome and the report. boolean false --skip_mycobrowser Skip Mycobrowser functional categories. Auto-enabled when --genus is not Mycobacterium and a custom reference is set. boolean false Their effect on the output tree is covered under outputs." -}, -{ -"page": "Parameters", -"section": "Annotation", -"url": "user-guide/parameters.html#annotation", -"text": "--genus Genus passed to Bakta. string \"Mycobacterium\" --species Species passed to Bakta. string \"Mycobacterium tuberculosis\" Bakta uses these to pick naming conventions and reference sets, so setting them correctly improves the baseline annotation that AMAP then corrects." -}, -{ -"page": "Parameters", -"section": "Resources", -"url": "user-guide/parameters.html#resources", -"text": "--max_cpus Ceiling on CPUs for any single process. integer the machine's CPUs --max_memory Ceiling on memory for any single process. string 16.GB --max_time Ceiling on wall-clock time for any single process. string 72.h --scratch Use node-local scratch for I/O-heavy processes. On by default in the HPC profiles. boolean false These are ceilings, not requests. Each process asks for what it needs and is then clamped to these values, so lowering them makes AMAP fit a smaller machine rather than making it run faster. The defaults are a single machine's. The HPC profiles raise them to 48 CPUs and 12" -}, -{ -"page": "Parameters", -"section": "Validation and information", -"url": "user-guide/parameters.html#validation-and-information", -"text": "--validate_only Run input validation and stop. boolean false --strict_validation Stop on validation warnings too, not only on errors. boolean false --help Print the help text and exit. boolean false --version Print pipeline and Nextflow versions and exit. boolean false" -}, -{ -"page": "Parameters", -"section": "Custom reference", -"url": "user-guide/parameters.html#custom-reference", -"text": "Supply both to annotate any prokaryote against your own trusted annotation. --ref_annotation Trusted reference annotation, GFF3 or GenBank. file null --ref_genome Trusted reference genome, FASTA. file null They must be given together; setting only one is an error. See custom reference for what AMAP does with them." -}, -{ -"page": "Parameters", -"section": "Reference and database arguments", -"url": "user-guide/parameters.html#reference-and-database-arguments", -"text": "You almost certainly do not need these Every file below ships inside the Docker and Singularity images, under /reference/. Running with -profile docker or -profile singularity picks them up automatically. They exist for running without a container, for swapping in updated reference files, or for adapting the pipeline by hand. --bakta_db Bakta database directory, inside the container or on the host. /data/bakta_db/db-light --eggnog_db Pre-formatted eggNOG database directory. eggnog_db --pathotyper_snps Lineage marker SNPs for pathotypr. /reference/snps_lineages.tsv --pathotyper_ref Reference FA" -}, -{ -"page": "Parameters", -"section": "Nextflow options worth knowing", -"url": "user-guide/parameters.html#nextflow-options-worth-knowing", -"text": "These belong to Nextflow rather than AMAP, so they take a single dash. -profile Selects execution profiles, comma separated: docker, singularity, slurm, sge, pbs, test. -resume Reuse cached results from the previous run. -work-dir Where intermediate files go. Defaults to ./work, and it gets large. -process.queue Override the scheduler queue or partition. -process.clusterOptions Extra flags passed to the scheduler, such as '--account=mygroup'. See running AMAP for how the profiles combine." -}, -{ -"page": "Running AMAP", -"section": "", -"url": "user-guide/running.html", -"text": "Nextflow runs the pipeline straight from GitHub, so there is nothing to clone and nothing to keep up to date by hand. nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv" -}, -{ -"page": "Running AMAP", -"section": "Profiles", -"url": "user-guide/running.html#profiles", -"text": "A profile is a bundle of settings. You normally combine one container profile with one executor profile. docker container Runs every process in the AMAP Docker image. singularity container Same image via Singularity or Apptainer. standard container No container. Every tool must already be on PATH. slurm executor Submits each process as a SLURM job. sge executor Submits to Sun Grid Engine or Univa Grid Engine. pbs executor Submits to PBS or Torque. lsf executor Submits to IBM Spectrum LSF. test data Small bundled dataset, for checking the installation. highmem resources Raises the ceilings to 4" -}, -{ -"page": "Running AMAP", -"section": "On a laptop or workstation", -"url": "user-guide/running.html#on-a-laptop-or-workstation", -"text": "nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv CPUs are auto-detected and memory is capped at 16 GB. Raise both if the machine allows: nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv --max_cpus 8 --max_memory 32.GB A handful of genomes is fine locally. Beyond about twenty, a cluster is worth the setup." -}, -{ -"page": "Running AMAP", -"section": "On an HPC cluster", -"url": "user-guide/running.html#on-an-hpc-cluster", -"text": "nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv Each process becomes its own scheduler job, with up to 200 running at once, so Bakta, Miniprot and eggNOG parallelise across all your samples. Defaults reach 48 CPUs and 128 GB per job, and every process carries a time limit so nothing sits in the queue forever. To match your site's queues and accounting: nextflow run PathoGenOmics-Lab/AMAP \\ -profile singularity,slurm \\ --tsv samples.tsv \\ --max_cpus 64 --max_memory 256.GB \\ -process.queue long \\ -process.clusterOptions '--account=mygroup' Dep" -}, -{ -"page": "Running AMAP", -"section": "Non-MTBC organisms", -"url": "user-guide/running.html#non-mtbc-organisms", -"text": "Lineage typing and Mycobrowser categories are specific to tuberculosis. Turn them off and bring your own reference: nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv \\ --skip_typing --skip_mycobrowser \\ --genus Escherichia --species \"Escherichia coli\" \\ --ref_annotation my_reference.gbk --ref_genome my_reference.fasta See custom reference." -}, -{ -"page": "Running AMAP", -"section": "Resuming", -"url": "user-guide/running.html#resuming", -"text": "Nextflow caches every completed task, so an interrupted run picks up where it stopped: nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \\ --tsv samples.tsv -resume Keep the parameters identical. Changing an input invalidates that task and everything downstream, which is correct behaviour but does mean the work is redone. The cache lives in the work directory, so do not delete it between a run and its resume." -}, -{ -"page": "Running AMAP", -"section": "Running only part of the pipeline", -"url": "user-guide/running.html#running-only-part-of-the-pipeline", -"text": "--skip_functional_annotation stops after the correction step. You get the corrected genes as a table with their proteins and sequences (*.AMAP.tsv, .faa, .fna) and the case tables, and skip eggNOG, duplications, the pangenome and the report. The GFF3 and GenBank files are written after eggNOG, so they are skipped too. Much faster when the corrected genes are all you are after: nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv --skip_functional_annotation" -}, -{ -"page": "Running AMAP", -"section": "What a run produces", -"url": "user-guide/running.html#what-a-run-produces", -"text": "Beyond the results themselves, results/pipeline_info/ holds the execution report, the timeline, the DAG, a trace with per-process resource usage, software_versions.yml with the exact version of every tool, and params.yml with the parameters used. Those last two are what you want when writing up methods." -}, -{ -"page": "Running AMAP", -"section": "Next steps", -"url": "user-guide/running.html#next-steps", -"text": "Outputs: what every file is Troubleshooting: when a run fails" -}, -{ -"page": "Outputs", -"section": "", -"url": "user-guide/outputs.html", -"text": "Everything lands under --outdir, results/ by default, in one subdirectory per pipeline stage. {sampleId} below is the sample's name as lineage typing leaves it, L4_G26689 for sample G26689, or the sample ID alone under --skip_typing." -}, -{ -"page": "Outputs", -"section": "Where to look first", -"url": "user-guide/outputs.html#where-to-look-first", -"text": "The whole analysis, interactive results/final_report.html Each genome and each gene, question by question results/amap_report.html Corrected annotation for a sample results/AMAP/{sampleId}/*-AMAP.gff3 Proteins for downstream tools results/AMAP/{sampleId}/*-AMAP.clean.faa Everything known about every gene results/AMAP/{sampleId}/*-AMAPmastertable.tsv Tool versions for your methods section results/pipeline_info/software_versions.yml" -}, -{ -"page": "Outputs", -"section": "The full tree", -"url": "user-guide/outputs.html#the-full-tree", -"text": "results/ โ”œโ”€โ”€ AMAP/ โ”‚ โ””โ”€โ”€ {sampleId}/ โ”‚ โ”œโ”€โ”€ *-AMAP.faa # Corrected proteins, all genes including pseudogenes โ”‚ โ”œโ”€โ”€ *-AMAP.clean.faa # No pseudogenes, no internal stops โ”‚ โ”œโ”€โ”€ *-AMAP.fna # Corrected nucleotide sequences โ”‚ โ”œโ”€โ”€ *-AMAP" -}, -{ -"page": "Outputs", -"section": "The key files", -"url": "user-guide/outputs.html#the-key-files", -"text": "" -}, -{ -"page": "Outputs", -"section": "*-AMAP.gff3 and *-AMAP.gbk", -"url": "user-guide/outputs.html#-amapgff3-and--amapgbk", -"text": "The corrected annotation, which is the point of the pipeline. The GFF3 is spec-compliant: gene features parent their CDS features, attributes are URL-escaped, ##sequence-region pragmas are emitted per contig, and eggNOG annotation rides along as Ontology_term, eC_number, COG_category, KEGG_ko, KEGG_Pathway and PFAMs. The sequences are appended after ##FASTA. Two kinds of gene carry marks, written the way NCBI writes them: Pseudogenes have pseudo=true in the GFF3 and /pseudo in the GenBank, with the reason in Note and /note: internal stop codons, a frameshift or lost stop that makes the frame r" -}, -{ -"page": "Outputs", -"section": "*-AMAP.faa versus *-AMAP.clean.faa", -"url": "user-guide/outputs.html#-amapfaa-versus--amapcleanfaa", -"text": "*-AMAP.faa mirrors the annotation exactly, pseudogenes included, so gene counts match the GFF3. *-AMAP.clean.faa drops the genes AMAP calls pseudogenes, those with pseudo=true in the GFF3, and any sequence with an internal stop codon, and strips the terminal stop. This is the file for orthology, phylogenomics, pangenome tools and dN/dS, all of which expect valid translations. It has fewer sequences than the GFF3 has genes, by design." -}, -{ -"page": "Outputs", -"section": "*-AMAPmastertable.tsv", -"url": "user-guide/outputs.html#-amapmastertabletsv", -"text": "One row per gene, joining the corrected coordinates, the Bakta annotation, the eggNOG assignment (COG category, description, preferred name, GO, EC, KEGG, PFAMs, CAZy) and, for MTBC, the Mycobrowser functional category. This is the table to load when you want to ask a question about the annotation." -}, -{ -"page": "Outputs", -"section": "*-AMAP.qc.tsv", -"url": "user-guide/outputs.html#-amapqctsv", -"text": "One row per sample: genome size, contig count, N50, GC%, total CDS, how many of them AMAP calls pseudogenes (those with pseudo=true in the GFF3), hypothetical protein percentage, mean and median gene length, coding density, and the eggNOG annotation rate. Useful for spotting the sample that went wrong." -}, -{ -"page": "Outputs", -"section": "CDS_CASES/", -"url": "user-guide/outputs.html#cds_cases", -"text": "The audit trail for the correction. Each case table lists the genes that fell into it, so you can see exactly what was changed and why. The correction algorithm explains the cases; *-summary.tsv aggregates the counts across samples and feeds the report. Its last three columns say which sample the counts are for: Sequence is the name its files carry (L4_G26689), Lineage what typing called it (Untyped under --skip_typing) and Sample its ID in the sample sheet (G26689)." -}, -{ -"page": "Outputs", -"section": "What a healthy summary looks like", -"url": "user-guide/outputs.html#what-a-healthy-summary-looks-like", -"text": "Two real runs, those of the tutorial, to calibrate expectations. On the left, M. tuberculosis CDC1551 against the bundled H37Rv reference. On the right, E. coli O157:H7 Sakai against K-12 MG1655, using a custom reference. case2, exact match 2813 3324 case3, length differs 24 31 case4, resolved conflict 1100 507 case1_kept, genuinely novel 11 1387 rv_missing, rescued 97 133 total_cds before correction 4077 5367 The one number that differs by two orders of magnitude is case 1, and that is biology: CDC1551 is the same species as the reference so almost nothing is genuinely new, while O157:H7 carr" -}, -{ -"page": "Outputs", -"section": "PANGENE/pangenome.Rtab", -"url": "user-guide/outputs.html#pangenepangenomertab", -"text": "The gene presence/absence matrix of the whole run, the one both reports draw the pangenome from: one row per gene, one column per sample, 1 where pangene found the gene in that sample's genome. Every genome is searched with the same proteins, the reference proteome and the genes no reference gene matches, so a gene with a 1 in every column is one all the samples share. A gene is named after its reference gene (Rv1506c) or, when no reference gene matches it, after one sample's copy of it (L4_CDC1551_01396, the ID in that sample's master table)." -}, -{ -"page": "Outputs", -"section": "final_report.html", -"url": "user-guide/outputs.html#final_reporthtml", -"text": "Self-contained, so it travels by email. Covers per-sample quality, missing reference genes by lineage, PE/PPE family distribution, KEGG pathways, duplication communities and network topology, the pangenome (core, softcore, shell, cloud, frequency spectrum, presence/absence heatmaps), COG and Mycobrowser distributions, case statistics and the associated tests, plus a searchable master-table explorer." -}, -{ -"page": "Outputs", -"section": "amap_report.html", -"url": "user-guide/outputs.html#amap_reporthtml", -"text": "One self-contained file, organised by the questions a run raises: how each genome annotated, where genes are missing or broken on the reference, one gene in every genome, what the correction changed, the pseudogenes by lineage, what lies outside the reference, and the functional profile. Written with or without the functional annotation. The interactive report goes through it." -}, -{ -"page": "Outputs", -"section": "Effect of the skip options", -"url": "user-guide/outputs.html#effect-of-the-skip-options", -"text": "--skip_typing: No PATHOTYPR/ directory. Samples keep their original IDs instead of being renamed by lineage. --skip_functional_annotation: The run stops after correction. No DUPLICATIONS/, PANGENE/ or final_report.html, and AMAP/ holds only the corrected table and sequences, *.AMAP.tsv, *.AMAP.faa and *.AMAP.fna, without the GFF3, the GenBank or the master table. You still get BAKTA/ and CDS_CASES/, including *_corrected.tsv with the corrected coordinates, and amap_report.html, without its eggNOG sections. --skip_mycobrowser: *-AMAP.MycoBrowser.tsv is written empty and the Mycobrowser sections" -}, -{ -"page": "Outputs", -"section": "Next steps", -"url": "user-guide/outputs.html#next-steps", -"text": "How it works: what produced all this Duplication metrics: reading the network numbers" -}, -{ -"page": "The report", -"section": "", -"url": "user-guide/report.html", -"text": "Every run ends in one file, results/final_report.html. It is self-contained: all the figures, tables and JavaScript are embedded, so it opens with no network and survives being emailed to someone who has never heard of Nextflow. To see what the sections look like before running anything, open the example report: a rendered report for twelve MTBC genomes across four lineages. The example uses generated data The twelve samples in it are synthetic. The distributions are drawn around a real M. tuberculosis CDC1551 run, so the shapes are representative, but no conclusion should be drawn from the nu" -}, -{ -"page": "The report", -"section": "What is in it", -"url": "user-guide/report.html#what-is-in-it", -"text": "" -}, -{ -"page": "The report", -"section": "Overview", -"url": "user-guide/report.html#overview", -"text": "Dataset composition, then a per-sample quality panel: CDS counts, correction rates and missing reference genes. The annotation quality tab carries coding density, hypothetical percentage and genome size, which is where an outlier sample declares itself before you have gone any further." -}, -{ -"page": "The report", -"section": "Annotation results", -"url": "user-guide/report.html#annotation-results", -"text": "CDS distribution per lineage, the genes with no counterpart in the reference, and the correction summary: how many genes fell into each case and what the adjudication did with them. Read this alongside the correction algorithm. Missing reference genes get both a heatmap, for spotting a gene absent from a whole lineage rather than one sample, and a searchable table." -}, -{ -"page": "The report", -"section": "Functional annotation", -"url": "user-guide/report.html#functional-annotation", -"text": "COG categories from eggNOG, Mycobrowser functional categories for MTBC, and a comparison against the reference distribution. Each has its own per-category breakdown in counts and percentages, with the statistical tests alongside. PE/PPE gene families get their own section, since they are both the hardest part of an MTBC genome to annotate and usually the most interesting. The pseudogenes close it: how many AMAP called in each sample, lineage by lineage, a sample with none at 0, the same genes the QC table counts and the GFF3 marks pseudo=true. The master table's note column says why each is a " -}, -{ -"page": "The report", -"section": "KEGG pathways", -"url": "user-guide/report.html#kegg-pathways", -"text": "The most represented metabolic pathways, and how they differ between lineages." -}, -{ -"page": "The report", -"section": "Duplications", -"url": "user-guide/report.html#duplications", -"text": "Community sizes, cross-genome homologous pairs and the network topology metrics. Duplication metrics explains what each number means." -}, -{ -"page": "The report", -"section": "Pangenome", -"url": "user-guide/report.html#pangenome", -"text": "Core, softcore, shell and cloud classification, the gene frequency spectrum, and presence/absence heatmaps." -}, -{ -"page": "The report", -"section": "CDS statistics", -"url": "user-guide/report.html#cds-statistics", -"text": "How many genes fall in each case, lineage by lineage: box and violin plots with Dunn's tests between lineages, the share of each case in every lineage, a heatmap of the means, and a table per case with the count, mean, median and spread of each lineage." -}, -{ -"page": "The report", -"section": "Master table explorer", -"url": "user-guide/report.html#master-table-explorer", -"text": "Every gene of every sample in one searchable, sortable table, with the functional annotation attached. Useful for answering a specific question without leaving the report." -}, -{ -"page": "The report", -"section": "What changes when steps are skipped", -"url": "user-guide/report.html#what-changes-when-steps-are-skipped", -"text": "--skip_typing: Everything still renders, but every sample is grouped under Untyped, so the per-lineage comparisons have a single group and the statistical tests drop out. --skip_mycobrowser: The Mycobrowser and PE/PPE sections are omitted. COG, KEGG, pangenome and duplications are unaffected. This is automatic for non-MTBC organisms. --skip_functional_annotation: No report at all. The run stops after correction." -}, -{ -"page": "The report", -"section": "Rendering it yourself", -"url": "user-guide/report.html#rendering-it-yourself", -"text": "The report is bin/report/report_AMAP.Rmd, rendered by the FINAL_REPORT process. To iterate on it without running the pipeline: python3 bin/report/demo/make_demo.py demo_inputs docker run --rm -v \"$PWD:/w\" -w /w paururo/amap:latest \\ Rscript -e 'rmarkdown::render(\"bin/report/report_AMAP.Rmd\", output_file=\"/w/demo.html\", params=list(version=\"v1.0.1\", cog_file=\"/w/demo_inputs/input_cog.txt\", Keep it self-contained Nothing in the Rmd, header.html or styles.css may point at a remote URL. pandoc will fetch and inline a remote resource when the machine rendering ha" -}, -{ -"page": "The interactive report", -"section": "", -"url": "user-guide/amap-report.html", -"text": "Next to final_report.html, every run now writes results/amap_report.html: one file that answers, section by section, the questions a run raises. Did each genome annotate well? Where on the reference are genes missing or broken? What happened to one gene in every genome? What did the correction change? Every table filters, sorts and saves as CSV, every figure saves as SVG, and a click on a genome or a gene opens it. It is written with or without the functional annotation. It needs only what the correction leaves, so a run with --skip_functional_annotation, which gets no final_report.html, still" -}, -{ -"page": "The interactive report", -"section": "The sections", -"url": "user-guide/amap-report.html#the-sections", -"text": "1. Did each genome annotate well? One row per genome: lineage, genome size, contigs and N50, Bakta's calls and AMAP's genes, the share of the reference genes it has, and how many are missing, pseudogenes, partial, found more than once or outside the reference. A value far from those of comparable genomes is flagged (see How a value is flagged). A click on a genome opens it: its numbers, and its genes in each state, each list searchable and saved as CSV. 2. Where on the reference are genes missing or broken? Every reference gene at its place on the reference, one row per genome, the genomes gro" -}, -{ -"page": "The interactive report", -"section": "Gene states", -"url": "user-guide/amap-report.html#gene-states", -"text": "Each reference gene gets one state in each genome, from the genes that name it in the genome's final annotation, AMAP//.AMAP.tsv. Intact A gene names it, and nothing below applies Missing No gene names it Pseudogene The best gene naming it is flagged as a pseudogene: an internal stop, a lost stop or a frame that runs on past the reference protein Partial The best gene naming it runs into a contig end and is cut there More copies More than one gene names it alone Length differs One intact copy, of another length than the reference gene Shared copy Only a gene that names several refe" -}, -{ -"page": "The interactive report", -"section": "How a value is flagged", -"url": "user-guide/amap-report.html#how-a-value-is-flagged", -"text": "Each genome is compared with its own lineage when that lineage has five genomes or more, and with every other genome otherwise. A value is flagged when it lies more than 3.5 robust standard deviations (1.4826 times the median absolute deviation) from their median, on the side that is bad: more missing genes, pseudogenes, partial genes, contigs or genes outside the reference, fewer reference genes found, a smaller N50, a genome size either way. Each also has to be further from the median than a minimum (five genes, five contigs, half a point of reference genes found, 1% of the genome size or 5%" -}, -{ -"page": "The interactive report", -"section": "What each section needs", -"url": "user-guide/amap-report.html#what-each-section-needs", -"text": ".AMAP.tsv FIX_CDS No report The reference proteome the reference No report Case summaries and tables TYPE_CASES No lineages, no correction section Bakta's folder BAKTA_ANNOTATOR No genome sizes, contigs or products of Bakta's calls Mycobrowser table the reference, MTBC only Names and products from the genomes' own annotation; no categories or drug resistance Master tables GET_OUTPUTS, functional annotation No COG categories or KEGG pathways Pangenome matrix REPORT_INPUTS, functional annotation No pangenome Homologous communities GET_ALLDUP, functional annotation No communities params.yml" -}, -{ -"page": "The interactive report", -"section": "Size", -"url": "user-guide/amap-report.html#size", -"text": "The data of each genome is a few kilobytes: its genes are stored as how far each lies from where the previous gene and the reference predict it, which for a genome collinear with its reference is mostly nothing. Up to 100 genomes, the report also lists every Bakta call AMAP did not keep, with its coordinates; past that it only counts them, to keep the file small, and CDS_CASES// has them for any run. 15 0.5 MB, measured 100 1.5 to 1.7 MB 300 1.9 to 2.1 MB 500 2.9 to 3.3 MB The sizes past fifteen are worked out from the fifteen MTBC genomes of eight lineages the report was measured on, th" -}, -{ -"page": "The interactive report", -"section": "Writing one by hand", -"url": "user-guide/amap-report.html#writing-one-by-hand", -"text": "bin/report/amap_report.py needs only Python 3.9 or later. Pointed at a results/ folder it reads the files where the pipeline publishes them: python3 bin/report/amap_report.py \\ --amap results/AMAP \\ --cases results/CDS_CASES \\ --bakta results/BAKTA \\ --master results/AMAP \\ --ref-proteome ref_proteome.faa \\ --mycobrowser Mycobacterium_tuberculosis_H37Rv_txt_v4.txt \\ --versions results/pipeline_info/software_ve The reference proteome is the one the run used: for H37Rv, the AMAP container has it at /reference/aa_H37Rv.translated.fas, and the Mycobrowser table at /refe" -}, -{ -"page": "Custom reference", -"section": "", -"url": "user-guide/custom-reference.html", -"text": "AMAP corrects an annotation by refereeing it against a trusted one. For MTBC that trusted annotation is H37Rv, bundled in the image. Point --ref_annotation and --ref_genome at your own and the same machinery works for any prokaryote. nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv \\ --skip_typing --skip_mycobrowser \\ --genus Escherichia --species \"Escherichia coli\" \\ --ref_annotation my_reference.gbk \\ --ref_genome my_reference.fasta Both must be given together. Setting one without the other stops the run immediately." -}, -{ -"page": "Custom reference", -"section": "What counts as a good reference", -"url": "user-guide/custom-reference.html#what-counts-as-a-good-reference", -"text": "A closed, well-curated genome of a close relative, ideally the type strain or a reference assembly from RefSeq. The correction assumes the reference genes are right, so its quality is the ceiling on your result. --ref_annotation: GFF3 (.gff, .gff3) or GenBank (.gbk, .gb, .genbank, .gbff). Detected from the extension, with GenBank tried first for anything else. --ref_genome: FASTA of the same genome. It must be the same assembly the annotation describes, since coordinates are taken at face value." -}, -{ -"page": "Custom reference", -"section": "What AMAP does with it", -"url": "user-guide/custom-reference.html#what-amap-does-with-it", -"text": "PREPARE_REFERENCE runs once, before any sample is processed, and does three things. 1. Extracts the reference proteome. Every CDS becomes a protein, headed GENE_ID_START-END contig=SEQUENCE, SEQUENCE being the name --ref_genome gives the sequence the gene is on. Those embedded coordinates are not decoration: the paralog step reads them back to predict where each protein belongs in the sample, which tells its own locus from a paralog's, so the header format matters. A CDS across the origin of a circular sequence ends past the sequence's last base, the way GFF3 writes it: tagA on plasmid pO157 (" -}, -{ -"page": "Custom reference", -"section": "What you must turn off", -"url": "user-guide/custom-reference.html#what-you-must-turn-off", -"text": "--skip_typing: pathotypr assigns MTBC lineages from tuberculosis-specific marker SNPs. Meaningless elsewhere, and it will fail. AMAP warns when --genus is not Mycobacterium and this is not set. --skip_mycobrowser: Mycobrowser functional categories are an MTBC resource. Auto-enabled when --ref_annotation is set and --genus is not Mycobacterium, but setting it explicitly is clearer. Also set --genus and --species to match your organism, since Bakta uses them for naming and reference selection." -}, -{ -"page": "Custom reference", -"section": "What stays the same", -"url": "user-guide/custom-reference.html#what-stays-the-same", -"text": "Everything else. Case classification, conflict resolution, missing-gene rescue, the stop codon check, eggNOG functional annotation, duplication analysis, the pangenome and the report all work identically. The report simply omits the lineage and Mycobrowser sections, and the pangenome calls each genome by its sample ID where a typed run would use its lineage name." -}, -{ -"page": "Custom reference", -"section": "What to expect from a real run", -"url": "user-guide/custom-reference.html#what-to-expect-from-a-real-run", -"text": "E. coli O157:H7 Sakai corrected against K-12 MG1655, both straight from RefSeq, in the tutorial's run: reference proteins extracted from the K-12 GenBank 4318 CDS Bakta called on that same reference genome 4296 Bakta false positives derived from the difference 142 case2 3324 case3 31 case4 The 142 are the automatically derived equivalent of the hand-curated MTBC false-positive list, and 1387 case 1 genes is what you would expect between two E. coli strains that genuinely differ. If " -}, -{ -"page": "Custom reference", -"section": "Worked example", -"url": "user-guide/custom-reference.html#worked-example", -"text": "E. coli K-12 MG1655 against the RefSeq reference: # Fetch the reference datasets download genome accession GCF_000005845.2 --include genome,gff3 unzip ncbi_dataset.zip nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv \\ --skip_typing --skip_mycobrowser \\ --genus Escherichia --species \"Escherichia coli\" \\ --ref" -}, -{ -"page": "Custom reference", -"section": "Things that trip people up", -"url": "user-guide/custom-reference.html#things-that-trip-people-up", -"text": "The annotation and the genome must match Coordinates from the annotation are used directly against the FASTA. An annotation from a different assembly of the same organism produces nonsense: extracted proteins will not translate, and the reference proteome will be mostly empty. PREPARE_REFERENCE prints the count of extracted proteins, so check it looks like a genome's worth. Reference genes need identifiers A GFF3 whose CDS features carry no locus_tag, ID, gene or Name is skipped entirely, and the run fails with \"No proteins extracted\". This happens with minimal GFF3 exports from some annotatio" -}, -{ -"page": "Custom reference", -"section": "Next steps", -"url": "user-guide/custom-reference.html#next-steps", -"text": "How it works: what the correction does with the reference Parameters: the full flag reference" -}, -{ -"page": "Overview", -"section": "", -"url": "how-it-works/index.html", -"text": "AMAP does not annotate genomes from scratch. It takes an annotation that already exists, compares it against one that is trusted, and adjudicates the differences. The premise is that de novo annotators are good at finding genes and less good at their exact boundaries. Bakta will call a start codon a few codons late, or split one gene into two, or invent a short ORF that is not really there. If a curated reference for a close relative exists, those errors are visible: align the reference proteins onto the new genome, and where the two disagree, decide which is more likely right. On MTBC genomes" -}, -{ -"page": "Overview", -"section": "The three inputs to a decision", -"url": "how-it-works/index.html#the-three-inputs-to-a-decision", -"text": "The Bakta annotation. Every CDS Bakta called on your genome, with coordinates and a locus tag. The Miniprot alignments. Every reference protein aligned onto your genome, with the coordinates where it landed. The reference proteins carry their own coordinates in their headers (Rv0001_1-1524), which lets a later step tell a real locus from a paralogous placement. The false-positive set. Proteins that a de novo annotator calls on the reference genome itself but that are not in the curated annotation. Bakta's characteristic mistakes, in other words. For MTBC this is a curated file; with a custom r" -}, -{ -"page": "Overview", -"section": "The pipeline", -"url": "how-it-works/index.html#the-pipeline", -"text": "Ten steps, from genomes to report: eggNOG database setup, skipped if it is already built Lineage typing with pathotypr, MTBC only Annotation with Bakta Protein alignment with Miniprot Case classification: every CDS sorted by how it relates to the reference Correction: conflicts resolved, missing genes rescued, stop codons checked Functional annotation with eggNOG-mapper Duplication analysis: within and across genomes Pangenome with pangene Report The workflow page walks through each with its inputs and outputs." -}, -{ -"page": "Overview", -"section": "The correction itself", -"url": "how-it-works/index.html#the-correction-itself", -"text": "Steps 5 and 6 are what makes AMAP more than a wrapper. Every CDS is sorted into one of five outcomes: Case 1 Bakta called it, the reference has nothing there. Potentially novel. Case 2 Coordinates match the reference exactly. Kept. Case 3 Coordinates match but alignment length differs. Kept. Case 4 Overlaps a reference gene, but the coordinates conflict. Must be resolved. Missing The reference has a gene there, Bakta missed it. Rescued. Cases 1 and 4 are where the work is: deciding which novel genes are real, and which set of coordinates wins a conflict. The correction algorithm gives the rule" -}, -{ -"page": "Overview", -"section": "In this section", -"url": "how-it-works/index.html#in-this-section", -"text": "The workflow: the ten steps in detail The correction algorithm: case classification, conflict resolution, quality control Duplication metrics: what the network numbers mean" -}, -{ -"page": "The workflow", -"section": "", -"url": "how-it-works/workflow.html", -"text": "The pipeline as a metro map: stations are the processes of main.nf, lines are what travels between them. Pick a station for what it does and where its files go, or switch an option to see what a run leaves out." -}, -{ -"page": "The workflow", -"section": "1. eggNOG database setup", -"url": "how-it-works/workflow.html#1-eggnog-database-setup", -"text": "Checks whether the eggNOG database exists at --eggnog_db and builds it if not. A bacteria.dmnd under 100 MB is treated as a truncated download and rebuilt. The database is large, so point --eggnog_db at persistent shared storage on a cluster. Skipped entirely under --skip_functional_annotation." -}, -{ -"page": "The workflow", -"section": "2. Lineage typing", -"url": "how-it-works/workflow.html#2-lineage-typing", -"text": "pathotypr assigns each genome to an MTBC lineage using phylogenetic marker SNPs, and the genomes are renamed after their sample ID with the lineage in front (L4_G26689 for sample G26689). That prefix carries through every downstream file and is what lets the report group results by lineage. pathotypr types each contig of a draft on its own, and the lineage of the draft is the one the markers of all its contigs add up to. A genome of one sequence has that sequence renamed after the sample and keeps the description that follows the name in its header, so Bakta still reads a closed genome as comp" -}, -{ -"page": "The workflow", -"section": "3. Annotation with Bakta", -"url": "how-it-works/workflow.html#3-annotation-with-bakta", -"text": "Bakta produces the baseline annotation: CDS, tRNA, rRNA and other features. This is the annotation AMAP will correct, and it is published unmodified under BAKTA/ so you can compare. --genus and --species are passed through, so setting them correctly improves the starting point." -}, -{ -"page": "The workflow", -"section": "4. Protein alignment with Miniprot", -"url": "how-it-works/workflow.html#4-protein-alignment-with-miniprot", -"text": "Miniprot aligns the reference proteome onto each genome, spliced and frameshift-aware. Two alignments are produced: the reference proteome, and the false-positive set that will filter novel-gene candidates. Output is PAF. Coordinates are converted from 0-based to 1-based on read, and for a protein hitting several positions, the best hit per position is kept so multi-copy families such as PE/PPE keep all their placements. In a draft assembly the genes a contig end cuts get a second pass. Miniprot places the larger piece of such a gene only, so what that placement lacks is aligned again as a pro" -}, -{ -"page": "The workflow", -"section": "5. Case classification", -"url": "how-it-works/workflow.html#5-case-classification", -"text": "Every Bakta CDS is compared against the alignments and sorted into cases 1 to 4, plus the reference genes Bakta missed. Comparisons are strand-aware throughout: a gene at (100, 200, +) and one at (100, 200, -) are different genes and are never merged. Tables land in CDS_CASES/, one per case, plus interactive overlap plots. The correction algorithm covers the classification." -}, -{ -"page": "The workflow", -"section": "6. Correction", -"url": "how-it-works/workflow.html#6-correction", -"text": "The cases are resolved into one corrected coordinate set: conflicts adjudicated, novel genes filtered, missing reference genes rescued, duplicates removed. Then check_cds.py translates every surviving CDS. A gene that does not end in a stop codon has its coordinates extended to the next in-frame stop, up to 300 bp. Genes with internal stops are flagged as pseudogenes rather than discarded. Anything that cannot be translated at all is dropped, so the annotation table and the FASTA files always agree. Outputs are the corrected gene table and its protein and nucleotide files under AMAP/ (*.AMAP.t" -}, -{ -"page": "The workflow", -"section": "7. Functional annotation", -"url": "how-it-works/workflow.html#7-functional-annotation", -"text": "eggNOG-mapper assigns COG categories, KEGG orthologs and pathways, GO terms, EC numbers, PFAM domains and CAZy families. This runs once over all samples together, not once per sample. With closely related genomes most proteins are identical across samples, so they are deduplicated first, annotated once, and the results propagated back. For a few hundred MTBC genomes that cuts the Diamond search by an order of magnitude and avoids hundreds of jobs contending for the same database on a shared filesystem. The results are merged into *-AMAPmastertable.tsv and into the GFF3 attributes." -}, -{ -"page": "The workflow", -"section": "8. Duplication analysis", -"url": "how-it-works/workflow.html#8-duplication-analysis", -"text": "Each proteome is searched against itself with Diamond to find within-genome duplications, which are grouped into families by connected components. Then the proteins of every sample's families are pooled and searched all against all to build homologous communities across genomes, with network metrics computed per community. Duplication metrics explains the numbers." -}, -{ -"page": "The workflow", -"section": "9. Pangenome", -"url": "how-it-works/workflow.html#9-pangenome", -"text": "pangene builds one pangenome graph for the whole run and a gene presence/absence matrix, which the report turns into core, softcore, shell and cloud classifications, a frequency spectrum and presence/absence heatmaps. Every genome is searched with the same proteins: the reference proteome, and the genes of each sample that no reference gene matches. miniprot aligns them to every sample's genome, its own included: the ones PATHOTYPR renamed or, with --skip_typing, the genomes of the sample sheet, each under its sample ID. A row of the matrix is then a gene, named after its reference gene or, ou" -}, -{ -"page": "The workflow", -"section": "10. Report", -"url": "how-it-works/workflow.html#10-report", -"text": "A single self-contained HTML file built from every output above. Sample quality, missing reference genes by lineage, PE/PPE families, KEGG pathways, duplication communities and topology, the pangenome, COG and Mycobrowser distributions, case statistics with the associated tests, and a searchable table of every gene. Alongside it, pipeline_info/ collects the Nextflow execution report, timeline, DAG and trace, plus software_versions.yml and params.yml for your methods section." -}, -{ -"page": "The workflow", -"section": "11. Interactive report", -"url": "how-it-works/workflow.html#11-interactive-report", -"text": "amap_report.html, one self-contained file organised by question: how each genome annotated, where reference genes are missing or broken, one gene in every genome, what the correction changed and the pseudogenes by lineage. It reads the correction's outputs directly, so it is written with or without the functional annotation, and the functional steps add their sections when they run. The interactive report goes through it." -}, -{ -"page": "The workflow", -"section": "Next steps", -"url": "how-it-works/workflow.html#next-steps", -"text": "The correction algorithm: the rules behind step 6 Outputs: every file this produces" -}, -{ -"page": "The correction algorithm", -"section": "", -"url": "how-it-works/correction-algorithm.html", -"text": "The core of AMAP. It referees between the initial Bakta annotation and the Miniprot alignments of a trusted reference proteome, classifying every candidate gene and then applying a set of rules to resolve the disagreements. Coordinates throughout are 1-based inclusive, so a gene's length is end - start + 1. Every comparison is strand-aware and contig-aware: genes on opposite strands at the same position are different genes and never conflict, and nor are genes on different contigs of a draft assembly, where every contig starts at 1. Bakta runs with --keep-contig-headers so that it names the co" -}, -{ -"page": "The correction algorithm", -"section": "0. Placements", -"url": "how-it-works/correction-algorithm.html#0-placements", -"text": "" -}, -{ -"page": "The correction algorithm", -"section": "Small proteins", -"url": "how-it-works/correction-algorithm.html#small-proteins", -"text": "Miniprot needs a seed of several residues and enough score to report an alignment, which a protein of a few dozen residues rarely gives it: annotating E. coli K-12 with its own proteome, it placed none of 33 proteins of 8 to 29 residues, and it put the toxin IbsC (b4665), 19 residues three of which differ from its neighbour IbsD's, on that neighbour only. Every protein without an exact, complete placement is looked for in the six-frame translation of the genome, preceded by a start codon and followed by a stop, and each exact copy found where Miniprot did not already place the protein becomes " -}, -{ -"page": "The correction algorithm", -"section": "Pieces of cut genes", -"url": "how-it-works/correction-algorithm.html#pieces-of-cut-genes", -"text": "A draft assembly cuts the genes its contig breaks run through. Miniprot reports only the placements within 5% of a protein's best, so it places the larger piece of such a gene and not the smaller one at the end of another contig: in CDC1551 cut into 40 contigs, none of the 25 smaller pieces was annotated. A placement was cut by its contig when it lacks residues at one end of its protein, ends within 30 bp of the contig's edge on that side, and those residues would not fit in the bases left. What it lacks, if 8 residues or more, is aligned again as a protein of its own, and its best placement i" -}, -{ -"page": "The correction algorithm", -"section": "Paralog placements", -"url": "how-it-works/correction-algorithm.html#paralog-placements", -"text": "Miniprot places a reference protein everywhere it scores within 5% of its best, so a protein with a paralog is placed at both loci. Before anything is classified, each placement is checked against where the protein belongs. The reference proteome carries each protein's coordinates in its header (Rv1150_1278269-1278820), but those are reference coordinates, and any indel upstream shifts every position after it. So each placement's reference position is predicted from the nearest proteins placed only once, which follow the sample's local offset from the reference, and the placement closest to wh" -}, -{ -"page": "The correction algorithm", -"section": "1. Classification", -"url": "how-it-works/correction-algorithm.html#1-classification", -"text": "Every Bakta CDS is compared against the Miniprot alignments. Case 2, perfect match. Same start, same end, same strand as an alignment, and the same length. Correct, kept as is. Case 3, imperfect match. Same coordinates, but the aligned length differs, typically from indels. Reliable enough, kept. Case 4, conflicting overlap. A Bakta CDS and an alignment overlap on the same strand, but their coordinates do not match. This is the conflict that has to be adjudicated. Case 1, potentially novel. A Bakta CDS with no overlap to any alignment. Either a genuine gene absent from the reference, or someth" -}, -{ -"page": "The correction algorithm", -"section": "2. Filtering case 1", -"url": "how-it-works/correction-algorithm.html#2-filtering-case-1", -"text": "Novel-gene candidates are scrutinised before being believed. A candidate is removed if either holds: it overlaps the false-positive set by 80% or more of its own length, meaning it is a call the annotator is known to make wrongly on the reference genome itself it aligns to the reference at 80% or more match identity, meaning it is not novel at all but a shifted version of a gene that is already there Candidates that pass both, and candidates with no reference alignment to judge by, are kept as case1_kept." -}, -{ -"page": "The correction algorithm", -"section": "3. Resolving case 4", -"url": "how-it-works/correction-algorithm.html#3-resolving-case-4", -"text": "" -}, -{ -"page": "The correction algorithm", -"section": "Single overlaps", -"url": "how-it-works/correction-algorithm.html#single-overlaps", -"text": "One Bakta CDS against one reference alignment, on the same strand of the same contig. Lengths are compared, and: the reference wins by default the Bakta version wins only if it is longer by more than 550 bp, and the alignment is not complete in the call's reading frame The asymmetry is deliberate. A reference alignment is usually the better boundary, but a much longer Bakta call normally means the alignment fragmented, and discarding a full-length gene in favour of a partial alignment would be the worse error. A complete alignment, one that covers the whole reference protein down to its stop c" -}, -{ -"page": "The correction algorithm", -"section": "Multiple overlaps", -"url": "how-it-works/correction-algorithm.html#multiple-overlaps", -"text": "Several Bakta CDS against one alignment, the signature of a gene that has been split. The best-overlapping Bakta CDS is picked first, then compared to the alignment on length: the reference wins if it is at least as long, if the Bakta CDS beats it by less than 500 bp, if the alignment is complete in the call's frame, or if the call overlaps other alignments as well, which makes it several reference genes read as one (Rv3022c and Rv3021c are one reading frame in CDC1551) otherwise the Bakta CDS wins" -}, -{ -"page": "The correction algorithm", -"section": "4. Deduplication on insert", -"url": "how-it-works/correction-algorithm.html#4-deduplication-on-insert", -"text": "Each winner is checked against the genes already accepted. Where a new coordinate overlaps an existing one by 70% or more on the same strand: the new gene displaces the existing one if it is longer by more than 550 bp it also displaces it if the existing gene is shorter by less than that margin and is not a reference gene otherwise the existing gene stands and the new coordinate is dropped A reference gene is one whose identifier is a reference protein's. The test used to look for coordinates embedded in the identifier, which are stripped as the alignments are read, so no gene qualified and a " -}, -{ -"page": "The correction algorithm", -"section": "5. Rescuing missing genes", -"url": "how-it-works/correction-algorithm.html#5-rescuing-missing-genes", -"text": "A reference protein is missing when no gene accounts for any of its placements, and then it goes back in at its best one. A gene accounts for a placement when it sits on it: same contig and strand, the same reading frame, covering at least 50% of it, and not known to be another reference gene. The frame matters because genes that really overlap sit in different frames (Rv0724A and Rv0725c); another reference gene in the same frame does not count either, since some are nested (E. coli hokC inside mokC). Placements on other genes' loci have already been dropped in step 0, so what is left to resc" -}, -{ -"page": "The correction algorithm", -"section": "6. Final deduplication", -"url": "how-it-works/correction-algorithm.html#6-final-deduplication", -"text": "Where one gene ends up at several coordinates, entries fully contained inside a different gene are dropped, always keeping at least one entry per gene so nothing disappears entirely. After the stop codon check, deduplicate_rvs.py ranks the genes that share a reference protein: a non-pseudogene over a pseudogene, then the longer one, then one Bakta also called at exactly those coordinates, then the one nearer where the protein belongs by synteny. The first keeps its place. The others stay as further copies of the gene when they are at least 90% as long as the first, clear of the copies kept, an" -}, -{ -"page": "The correction algorithm", -"section": "7. Stop codon check", -"url": "how-it-works/correction-algorithm.html#7-stop-codon-check", -"text": "check_cds.py translates every surviving CDS. It is given the reference proteome, which settles what a translation alone cannot. No terminal stop codon. Coordinates are extended to the next in-frame stop, searching 300 bp in the right direction for the strand, or as far as the reference protein says is missing when that is further. If the extension introduces internal stops or finds nothing, the original coordinates are kept and the gene is flagged as a pseudogene. An extension goes no further than the reference protein. A gene may be extended to a stop at most 10 residues past the end of its r" -}, -{ -"page": "The correction algorithm", -"section": "Threshold summary", -"url": "how-it-works/correction-algorithm.html#threshold-summary", -"text": "False-positive overlap case 1 filtering 80% Reference match identity case 1 filtering 80% Bakta length advantage single overlaps 550 bp Bakta length advantage multiple overlaps 500 bp Overlap for deduplication on insert 70% Length margin for displacement on insert 550 bp Distance from the predicted position paralog placements 10 kb Proteins placed once, each side, for the local offset paralog placements 5 Cover to claim another protein's placement paralog placements 50% Shortest protein looked for exactly small proteins 8 residues Longest protein looked for with substitutions small proteins 50" -}, -{ -"page": "The correction algorithm", -"section": "Next steps", -"url": "how-it-works/correction-algorithm.html#next-steps", -"text": "The workflow: where this sits in the pipeline Outputs: the case tables that record every decision" -}, -{ -"page": "Duplication metrics", -"section": "", -"url": "how-it-works/duplications.html", -"text": "AMAP looks at gene duplication twice: within each genome, and across the whole set." -}, -{ -"page": "Duplication metrics", -"section": "How the families are found", -"url": "how-it-works/duplications.html#how-the-families-are-found", -"text": "Each corrected proteome is searched against itself with Diamond, excluding self-hits. A pair is kept when the identical residues of the alignment cover 80% or more of either the query or the subject, so a short domain match between two otherwise unrelated proteins is not counted as duplication. Kept pairs become links in a graph, and its connected pieces become the duplication families, listed in DUPLICATIONS/{sample}/duplications_{sample}_groups.tsv. Because the rule takes the larger of the two shares, a fragment joins the family of the gene it came from: a match that covers almost all of a s" -}, -{ -"page": "Duplication metrics", -"section": "How the communities are built", -"url": "how-it-works/duplications.html#how-the-communities-are-built", -"text": "The proteins of every sample's families are then pooled and searched all against all. Within each genome, the pairs that pass the same rule make a network, which the Louvain method splits into communities, with a fixed seed so that a rerun splits it the same way. Two communities are homologous when at least one hit joins them, and each homologous pair is one row of DUPLICATIONS/homologous_communities.tsv, with an interactive network in the matching HTML file." -}, -{ -"page": "Duplication metrics", -"section": "The columns", -"url": "how-it-works/duplications.html#the-columns", -"text": "A row describes two communities: the columns ending in 1 are the first, those ending in 2 the second. The first is the one whose sample comes first by name, or the one with the lower number when both are in the same sample, and the rows come in that order too, so a rerun writes the same file. For each: Genome, Community: the sample, and the community's number in it. Size: how many proteins it holds. Elements: the eggNOG COG categories of its proteins, with how many have each, the largest first. A protein eggNOG gave no category counts as -, eggNOG's mark for none. AvgDegree: links per protein," -}, -{ -"page": "Duplication metrics", -"section": "Reading them together", -"url": "how-it-works/duplications.html#reading-them-together", -"text": "Density and clustering at 1 mean copies that all match each other: a recent expansion of near-identical copies. In CDC1551 these are the transposases of insertion sequences, the ESAT-6-like Esx proteins and PE proteins. Lower density and longer paths mean copies that have drifted apart, each still matching some of the others but not all, as M. bovis's copies of the transposases above have. Or, less happily, an artefact of assembly fragmentation, where one gene broken across contigs looks like several partial copies. Cross-checking against the contig count in *-AMAP.qc.tsv tells the two apart. " -}, -{ -"page": "Duplication metrics", -"section": "Next steps", -"url": "how-it-works/duplications.html#next-steps", -"text": "Outputs: where the duplication files live The workflow: where this step sits" -}, -{ -"page": "Containers", -"section": "", -"url": "deployment/containers.html", -"text": "Every tool AMAP calls lives in one image, paururo/amap, published on Docker Hub. Both the docker and singularity profiles use it, so nothing needs installing beyond a container engine. nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv nextflow run PathoGenOmics-Lab/AMAP -profile singularity --tsv samples.tsv Nextflow pulls the image on first use and caches it. Singularity converts it to a .sif on the way, which takes a few minutes once." -}, -{ -"page": "Containers", -"section": "What is inside", -"url": "deployment/containers.html#what-is-inside", -"text": "Tools: Bakta 1.12.0, Miniprot 0.18, Diamond 2.1.24, eggNOG-mapper 2.1.13, pangene 1.1, pathotypr 1.0.0, gfatools. Runtimes: Python 3.9 with pandas, NumPy, Biopython, matplotlib, plotly, NetworkX, intervaltree, pyvis and python-louvain. R 4.4 with the report packages. Bundled data under /reference/, so a standard MTBC run needs no downloads: MTB_ancestor_reference.fasta MTBC ancestral reference genome aa_H37Rv.translated.fas H37Rv reference proteome bad.bakta.translated.fas Curated Bakta false positives new_refH37Rv.fasta H37Rv reference genome sequence.gb H37Rv GenBank, for Bakta Mycobacterium" -}, -{ -"page": "Containers", -"section": "Platform", -"url": "deployment/containers.html#platform", -"text": "The image is linux/amd64 only, and every Dockerfile stage pins it. On Apple Silicon it runs under emulation, which works but is slow. For anything past a couple of genomes, use an amd64 machine or a cluster." -}, -{ -"page": "Containers", -"section": "Pinning a version", -"url": "deployment/containers.html#pinning-a-version", -"text": "latest tracks the newest release. For a reproducible analysis, pin the version in nextflow.config or on the command line: nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv \\ -process.container 'paururo/amap:1.0.1' results/pipeline_info/software_versions.yml records the exact version of every tool a run used, which is what belongs in a methods section." -}, -{ -"page": "Containers", -"section": "Building it yourself", -"url": "deployment/containers.html#building-it-yourself", -"text": "Only needed if you are changing what is in the image. git clone https://github.com/PathoGenOmics-Lab/AMAP cd AMAP docker build --platform linux/amd64 -f docker/Dockerfile -t amap:dev . The build takes about an hour: it downloads the Bakta database and solves a conda environment with R. Check the result before using it: docker/smoke_test.sh amap:dev That verifies the tools, the Python and R libraries, the bundled reference data, and runs the repository test suite with the image interpreter. The last part matters, since a library update can break the pipeline scripts without the image looking an" -}, -{ -"page": "Containers", -"section": "Singularity on a cluster", -"url": "deployment/containers.html#singularity-on-a-cluster", -"text": "Set a shared cache so every job reuses one converted image instead of each building its own: export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache Put it in your shell profile or the launch script. Without it, a large run can have dozens of jobs converting the same image at once. More cluster specifics in deployment on HPC." -}, -{ -"page": "Containers", -"section": "Next steps", -"url": "deployment/containers.html#next-steps", -"text": "HPC deployment: scratch, shared filesystems, schedulers Troubleshooting: when a run fails" -}, -{ -"page": "HPC clusters", -"section": "", -"url": "deployment/hpc.html", -"text": "Past a couple of dozen genomes, a scheduler is worth the setup. Combine a container profile with an executor profile: nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv" -}, -{ -"page": "HPC clusters", -"section": "The executor profiles", -"url": "deployment/hpc.html#the-executor-profiles", -"text": "slurm SLURM 100 per minute sge Sun Grid Engine, Univa 20 per minute pbs PBS, Torque 20 per minute lsf IBM Spectrum LSF 20 per minute All four queue up to 200 jobs at once, default to 48 CPUs and 128 GB per job, and enable node-local scratch. SGE additionally requests memory through -l h_vmem, and uses the smp parallel environment. Tune the concurrency to your site's policy if 200 is too aggressive: nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \\ --tsv samples.tsv -process.queueSize 50" -}, -{ -"page": "HPC clusters", -"section": "Fitting your site", -"url": "deployment/hpc.html#fitting-your-site", -"text": "nextflow run PathoGenOmics-Lab/AMAP \\ -profile singularity,slurm \\ --tsv samples.tsv \\ --max_cpus 64 --max_memory 256.GB \\ -process.queue long \\ -process.clusterOptions '--account=mygroup' -process.queue sets the partition or queue, and -process.clusterOptions passes anything else straight through, which is where accounting flags, QoS and node constraints go. On SLURM, Bakta's jobs, the longest a run sends, also ask for --qos=medium. The slurm profile sets that for Bakta alone, and for Bakta it takes the place of -process.clusterOptions, so give Bakta your site's options in" -}, -{ -"page": "HPC clusters", -"section": "Scratch", -"url": "deployment/hpc.html#scratch", -"text": "--scratch is on by default in every HPC profile, and it matters. Without it, every process reads and writes directly on the shared filesystem, and Bakta and eggNOG in particular generate enough metadata traffic to slow a cluster down for everyone. With it, work happens on node-local disk and only the results are copied back. Turn it off only if your nodes have no local disk: nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \\ --tsv samples.tsv --scratch false" -}, -{ -"page": "HPC clusters", -"section": "Where to put things", -"url": "deployment/hpc.html#where-to-put-things", -"text": "The Singularity cache. Set it once, somewhere shared: export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache Otherwise the singularity profile caches under --outdir, and a large run can have many jobs converting the same image at once. The eggNOG database. Large, built once, reused forever. Point it at persistent shared storage rather than somewhere per-run: nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \\ --tsv samples.tsv --eggnog_db /shared/databases/eggnog The work directory. It gets large, tens of gigabytes for a big run. Put it on a filesystem with room and a gener" -}, -{ -"page": "HPC clusters", -"section": "Launching", -"url": "deployment/hpc.html#launching", -"text": "Nextflow's own process must survive the whole run, which will outlive an SSH session. Either run it under tmux or screen, or submit it as a small job of its own: #!/bin/bash #SBATCH --job-name=amap #SBATCH --cpus-per-task=2 #SBATCH --mem=8G #SBATCH --time=48:00:00 #SBATCH --output=amap_%j.log module load singularity export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache nextflow run PathoGenOmics-Lab/AMAP \\ -profile singularity,slurm \\ --tsv sam The launcher itself needs very little: it submits and waits while the real work happens in the jobs it spawns. Give it plenty of wall tim" -}, -{ -"page": "HPC clusters", -"section": "Memory and retries", -"url": "deployment/hpc.html#memory-and-retries", -"text": "A process killed by the out-of-memory killer is resubmitted with more memory, scaling with the attempt number up to --max_memory. That handles the occasional unusually large genome without you having to size everything for the worst case. eggNOG is the memory-hungry step. It runs once over all samples rather than once per sample, which keeps a shared filesystem from being hammered but does mean one large job. The highmem profile exists for this: nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm,highmem \\ --tsv samples.tsv" -}, -{ -"page": "HPC clusters", -"section": "Watching a run", -"url": "deployment/hpc.html#watching-a-run", -"text": "squeue -u $USER shows the jobs. When it finishes, pipeline_info/ holds the execution report, the timeline, and a trace with per-process CPU, memory and runtime, which is what to look at before adjusting resources for the next run." -}, -{ -"page": "HPC clusters", -"section": "Next steps", -"url": "deployment/hpc.html#next-steps", -"text": "Troubleshooting: when something fails Containers: what the image holds" -}, -{ -"page": "Troubleshooting", -"section": "", -"url": "deployment/troubleshooting.html", -"text": "" -}, -{ -"page": "Troubleshooting", -"section": "Reading a failure", -"url": "deployment/troubleshooting.html#reading-a-failure", -"text": "When a process fails, Nextflow prints the work directory for the failed task. That directory has everything you need: work/ab/cd1234.../ โ”œโ”€โ”€ .command.sh # the exact script that ran โ”œโ”€โ”€ .command.log # everything it printed โ”œโ”€โ”€ .command.err # stderr on its own โ””โ”€โ”€ .exitcode # what it exited with .command.err is almost always where the answer is. Re-running .command.sh inside the container reproduces the failure by hand. results/pipeline_info/execution_trace_*.txt shows every task with its status, CPU, memory and runtime, which is the fastest way to see whether a failure was r" -}, -{ -"page": "Troubleshooting", -"section": "Before the pipeline starts", -"url": "deployment/troubleshooting.html#before-the-pipeline-starts", -"text": "\"Unexpected input: '('\" in nextflow.config: Nextflow 25.10 and later read pipelines with a stricter parser, which older copies of AMAP do not pass. Nextflow keeps the copy it downloaded first, so fetch the current one with nextflow pull PathoGenOmics-Lab/AMAP and run again. To run an older AMAP release as it is, export NXF_SYNTAX_PARSER=v1 first. \"Input TSV file not found\": The path in --tsv is wrong, or relative to somewhere other than where you launched Nextflow. \"Invalid TSV format\": The header must be exactly sampleId and fasta, separated by a real tab. Spaces are the usual culprit. cat -t" -}, -{ -"page": "Troubleshooting", -"section": "Container problems", -"url": "deployment/troubleshooting.html#container-problems", -"text": "The image will not pull. Check network access from the compute nodes, which often differs from the login node. Pre-pulling on the login node and setting NXF_SINGULARITY_CACHEDIR sidesteps it: export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache singularity pull docker://paururo/amap:latest Slow on Apple Silicon. The image is amd64 only, so it emulates. Fine for the test profile, painful beyond that. Use an amd64 machine. Docker permission denied. Your user is not in the docker group. Either add it, or use Singularity." -}, -{ -"page": "Troubleshooting", -"section": "Memory", -"url": "deployment/troubleshooting.html#memory", -"text": "A process is killed with exit 137 or 140. Out of memory. AMAP retries with more, scaling by attempt, so a single occurrence in the log that later succeeds is normal. Persistent failures mean raising the ceiling: --max_memory 256.GB On a cluster, that is the whole answer. On a macOS or Windows laptop it is the wrong one: Docker Desktop runs containers inside a VM with its own memory ceiling, commonly 8 GB, and that ceiling is what killed the process. Nextflow sized the request against the host and never saw it. Raising --max_memory only makes the request larger than a limit it was already over." -}, -{ -"page": "Troubleshooting", -"section": "Filesystem", -"url": "deployment/troubleshooting.html#filesystem", -"text": "\"No space left on device\". Usually the work directory, which grows to tens of gigabytes. Move it somewhere with room: -work-dir /scratch/$USER/amap_work Also check your $TMPDIR quota, since Diamond and Bakta both use it heavily. Everything is slow on a shared filesystem. Confirm --scratch is on. It defaults to true in the HPC profiles, but a standard or docker profile on a cluster will not have it. Cache misses on resume. NFS timestamp drift across nodes invalidates Nextflow's cache. The slurm profile already sets process.cache = 'lenient' for this. If you are using a different executor, add i" -}, -{ -"page": "Troubleshooting", -"section": "The eggNOG database", -"url": "deployment/troubleshooting.html#the-eggnog-database", -"text": "It downloads every run. --eggnog_db is pointing somewhere per-run. Give it persistent shared storage. It rebuilds itself unexpectedly. A bacteria.dmnd under 100 MB is treated as a truncated download and rebuilt. Usually means an earlier download was interrupted. \"has eggnog_db.ok, so EGGNOG_DB will not run again\". eggnog_db.ok tells Nextflow the database is built, so it never runs EGGNOG_DB again, but what it needs is not all there. Delete the files the message names and the next run builds it again." -}, -{ -"page": "Troubleshooting", -"section": "Results that look wrong", -"url": "deployment/troubleshooting.html#results-that-look-wrong", -"text": "Far fewer genes than expected. Check the assembly. A fragmented assembly loses genes across contig boundaries, and they turn up in *-rvmissing.tsv rather than being corrected. *-AMAP.qc.tsv has the contig count and N50. \"Genes lost to (Start, End, Strand, Contig) collisions\" in the log. Two of the classification's sources listed a gene at exactly the same place, and one entry was kept. Genes on different contigs never collide, since the contig is part of the key. *-AMAP.clean.faa has fewer sequences than the GFF3 has genes. Expected. The clean proteome drops pseudogenes and internal-stop seque" -}, -{ -"page": "Troubleshooting", -"section": "Still stuck", -"url": "deployment/troubleshooting.html#still-stuck", -"text": "Open an issue at github.com/PathoGenOmics-Lab/AMAP/issues with: the command you ran the failing task's .command.err results/pipeline_info/software_versions.yml results/pipeline_info/params.yml The last two pin down the exact versions and parameters, which is usually what the answer turns on." -}, -{ -"page": "Changelog", -"section": "", -"url": "about/changelog.html", -"text": "All notable changes to AMAP are documented in this file." -}, -{ -"page": "Changelog", -"section": "[Unreleased]", -"url": "about/changelog.html#unreleased", -"text": "Annotating H37Rv against its own reference now gives back every CDS of the AL123456.3 GenBank record at exactly its coordinates, strand and name: 4031 of 4031, against 4020 before. The changes apply to every genome, so results are not directly comparable with 1.0.1." -}, -{ -"page": "Changelog", -"section": "๐Ÿ”ฌ Correctness", -"url": "about/changelog.html#-correctness", -"text": "Paralog copies no longer take another gene's place: a protein with a paralog was placed at both loci, and which copy survived came down to insertion order. On H37Rv itself that lost Rv1150, Rv1945 and Rv2828c; on M. bovis it reported genes the strain lacks (the mce3 operon of RD7, the phiRv2 prophage of RD11, plcBC of RD5) as copies sitting on their paralogs. Each placement is now checked against where its protein belongs, predicted from the sample's own offset to the reference so that indels do not throw it off Strand of a resolved conflict: taken from the protein's first placement anywhere i" -}, -{ -"page": "Changelog", -"section": "๐Ÿ› Pipeline", -"url": "about/changelog.html#-pipeline", -"text": "Pseudogenes in the GFF3 and GenBank: the GFF3 never said which genes were pseudogenes, and the GenBank only for genes with an internal stop, though the master table knew. Both now carry AMAP's flag with its reason (pseudo=true and Note, /pseudo and /note), and mark the ends a contig cut off (partial=true with start_range or end_range, < or >); the master table and *-AMAP.tsv have them as their last columns The default H37Rv reference under Docker: its files live inside the container and were handed to processes as paths, so Nextflow mounted /reference from the host, which Docker Desktop refuse" -}, -{ -"page": "Changelog", -"section": "๐Ÿ“Š Report", -"url": "about/changelog.html#-report", -"text": "A report organised by question, results/amap_report.html: next to final_report.html, a new process, AMAP_REPORT, writes one self-contained file from each genome's own outputs: a table of the genomes with the values far from comparable genomes flagged, a map of every reference gene in every genome where regions of difference show as blocks, a lookup of one gene in every genome, the correction from Bakta's calls to AMAP's genes with the genes behind each step, the pseudogenes by lineage, what lies outside the reference, and the functional profile, the categories that differ most between lineages" -}, -{ -"page": "Changelog", -"section": "๐Ÿงน Code", -"url": "about/changelog.html#-code", -"text": "reconcile_coordinates is gone: it read reference coordinates from protein identifiers that have not carried them since March 2026, so it changed nothing. The paralog check does its job with the sample's own offsets" -}, -{ -"page": "Changelog", -"section": "๐Ÿ“š Documentation", -"url": "about/changelog.html#-documentation", -"text": "A website in plain HTML, drawn with real genes: MkDocs is gone. The front page starts a newcomer in three steps and builds the command for their machine, the pipeline is a metro map whose stations are main.nf's processes, and the four cases, the missing genes and the harder cases (paralogs, small proteins, contig ends, frameshifts, copies) are drawn on genes of CDC1551, M. bovis and E. coli from the benchmark. The tutorial and user guide stay markdown, readable on GitHub, and tools/build_docs.py turns them into pages of the same site, dark by default The site is checked against the code: tools" -}, -{ -"page": "Changelog", -"section": "๐Ÿงช Testing", -"url": "about/changelog.html#-testing", -"text": "tests/selftest/h37rv_selftest.sh annotates H37Rv with itself inside the container and fails unless every GenBank CDS comes back exactly tests/test_cache_keys.py fails when a process runs AMAP code that is not in its cache key, or when its script names a bin/ entry as a bare word without declaring it tests/benchmark/benchmark.sh scores the correction algorithm on seven genomes whose right answer is known (H37Rv twice, CDC1551 complete and in 40 contigs, M. bovis, E. coli K-12 and Sakai), downloading them and running Bakta and miniprot in the container the first time, and lists every gene that c" -}, -{ -"page": "Changelog", -"section": "[1.0.1] - 2026-07-25", -"url": "about/changelog.html#101---2026-07-25", -"text": "Bug fixes, most of them found by running the pipeline on real genomes rather than by reading it. Several changed what the pipeline outputs, so results from 1.0.0 are not directly comparable." -}, -{ -"page": "Changelog", -"section": "๐Ÿ”ฌ Correctness", -"url": "about/changelog.html#-correctness-1", -"text": "Genes no longer vanish during overlap resolution: _insert_with_dedup queued entries for eviction while scanning for conflicts, and applied them even when it then abandoned the insert, so genes were deleted with nothing put in their place Reference pseudogenes are recognised for any organism: the check was gated on the identifier starting with Rv, so on every other organism every reference pseudogene was extended to the next stop and lost its flag. The same gene came out differently depending on what it was called The extracted reference proteome carries its stop codon: extract_ref_proteins.py " -}, -{ -"page": "Changelog", -"section": "๐Ÿ› Crashes", -"url": "about/changelog.html#-crashes", -"text": "--skip_mycobrowser and organism-agnostic mode: the placeholder frame was built with a duplicate Locus column, so pd.merge refused to run and GET_OUTPUTS failed on every run without Mycobrowser Miniprot PAF parsing: pandas inferred the column count from the first row, and miniprot varies its trailing tags per alignment, so any file whose first row was narrower aborted the parse Biopython 1.85 and newer: SeqFeature.strand was removed; to_panaroo.py and extract_ref_proteins.py now read it off the location -profile test: pointed at a sample sheet naming paths that are not in this repository. It no" -}, -{ -"page": "Changelog", -"section": "๐Ÿ“‰ Silent data loss", -"url": "about/changelog.html#-silent-data-loss", -"text": "First Diamond hit: read_and_create left header inference on while stripping # comments, so the first alignment became the column names and disappeared from the duplication analysis Multi-part gene names in rename_fastas.py: headers were cut at the third underscore, collapsing every PE/PPE gene of a sample onto one identifier eggNOG annotation rate: counted with notna(), and emapper writes - for fields it could not fill, so the QC metric read close to 100% whatever the run had done" -}, -{ -"page": "Changelog", -"section": "โšก Performance", -"url": "about/changelog.html#-performance", -"text": "reconcile_coordinates is linear rather than quadratic: it scanned every protein and every reference for each gene, calling a regex on each candidate. Indexed by gene name up front, a 4000-gene genome goes from 18.3 s to 0.011 s with identical output over 200 randomised genomes deduplicate: list.count() per element and a full scan per duplicated gene, replaced with a Counter and a single grouping pass" -}, -{ -"page": "Changelog", -"section": "๐Ÿ“Š Report", -"url": "about/changelog.html#-report-1", -"text": "Layout: the stylesheet was losing to the one rmarkdown inlines afterwards, so the article sat underneath the table of contents and the left logo on top of the title Self-contained without a network at render time: jQuery came from a CDN and the second logo from GitHub. pandoc inlines those when the render machine has a network and leaves them dangling when it does not, so a report rendered on an isolated node arrived with a broken image Half the size: 8.2 MB to 4.4 MB, by shipping only the plotly traces actually used and quantising the figures The version in the report is the version that prod" -}, -{ -"page": "Changelog", -"section": "๐Ÿ“š Documentation and infrastructure", -"url": "about/changelog.html#-documentation-and-infrastructure", -"text": "Documentation site built with MkDocs Material, published to GitHub Pages, including a tutorial with the numbers from real runs and a rendered example report Docker image workflow: builds, smoke tests and publishes on a v* tag or on demand Tests workflow enabled: it had been sitting disabled and would have failed, since its dependency list was missing plotly and its syntax check had drifted five files behind requirements-test.txt: one dependency list for CI, the docs and a local checkout. environment.yml was missing the Python plotly, so a conda environment built from it could not start classif" -}, -{ -"page": "Changelog", -"section": "[1.0.0] - 2026-04-01", -"url": "about/changelog.html#100---2026-04-01", -"text": "" -}, -{ -"page": "Changelog", -"section": "๐Ÿ”ฌ Correctness", -"url": "about/changelog.html#-correctness-2", -"text": "Strand-aware gene dictionary: combined_dict keys changed from (Start, End) to (Start, End, Strand): genes on opposite strands at the same coordinates no longer collide silently Consistent 1-based lengths: all length calculations now use End - Start + 1 (15 sites fixed) Robust ID cleaning: regex _\\d+-\\d+$ replaces brittle 4-part underscore rule, so PE_PGRS genes are handled correctly Exact gene name matching: prevents false matches (Rv3508 โ‰  Rv3508c, Rv1 โ‰  Rv10) IntervalTree half-open intervals: all 9 operations corrected from [Start:End] to [Start:End+1] Cross-strand gene preservation: merge o" -}, -{ -"page": "Changelog", -"section": "๐Ÿ›ก๏ธ Robustness", -"url": "about/changelog.html#-robustness", -"text": "Pre-flight validation: new VALIDATE_INPUTS process checks FASTA format, N-content, GC%, genome size, duplicate headers and database paths, before any computation --validate_only mode: check all inputs without executing the pipeline eggNOG DB integrity: bacteria.dmnd <100 MB detected as corrupted โ†’ automatic rebuild Bakta DB pre-check: actionable error message with download command OOM retry: memory scales with task.attempt (e.g., 50 GB โ†’ 100 GB โ†’ 150 GB) 0-protein handling: eggNOG batch handles empty inputs gracefully Completion handler: prints duration, succeeded/failed/ignored counts" -}, -{ -"page": "Changelog", -"section": "๐Ÿ“‹ Reproducibility", -"url": "about/changelog.html#-reproducibility", -"text": "nextflow.manifest: version 1.0.0, Nextflow โ‰ฅ23.04.0, DOI, homepage SOFTWARE_VERSIONS: YAML + TSV with exact versions of 12 tools DUMP_PARAMS: complete parameter dump per run Diamond sorted output: deterministic results regardless of thread count environment.yml: Conda environment with pinned versions Louvain seed: fixed to 42 for reproducible community detection" -}, -{ -"page": "Changelog", -"section": "โšก Performance & HPC", -"url": "about/changelog.html#-performance--hpc", -"text": "Dynamic resource scaling: memory grows with each retry attempt Node-local scratch: --scratch true (default on SLURM/SGE/PBS/LSF) Diamond block size tuning: auto-calculated from available RAM Bakta local temp: --tmp-dir on node-local disk (avoids NFS metadata storms) eggNOG batch mode: single run for all samples (10ร— faster, 50ร— less filesystem I/O) Thread utilization: Bakta 12, Diamond 6-24, Miniprot 2ร—3, Pangene 2ร—N SLURM/SGE/PBS/LSF profiles: with time limits, queue sizes, rate limiting storeDir for eggNOG DB: persists across runs, survives nextflow clean" -}, -{ -"page": "Changelog", -"section": "๐Ÿ“ Scalability", -"url": "about/changelog.html#-scalability", -"text": "find -print0 | xargs -0: replaces echo|tr|grep in REPORT_INPUTS (handles 500+ files) publishDir mode: 'link': hardlinks instead of copies for per-sample outputs cache 'lenient': tolerates NFS timestamp drift between cluster nodes awk close(): prevents file handle exhaustion on 500+ sample eggNOG splits JVM tuning docs: NXF_OPTS guidance for large runs" -}, -{ -"page": "Changelog", -"section": "๐Ÿงฌ Organism-Agnostic", -"url": "about/changelog.html#-organism-agnostic", -"text": "Custom reference: --ref_annotation + --ref_genome replaces built-in H37Rv Auto-detect mode: --skip_mycobrowser auto-enabled when genus โ‰  Mycobacterium Dynamic reference comparison: report uses median distribution (not hardcoded H37Rv) Auto-extending color palette: non-MTBC lineages get colors automatically Generic labels: \"Ratio to Median\" instead of \"Ratio to H37Rv\" Manifest updated: \"Assisted Microbial Annotator Pipeline\"" -}, -{ -"page": "Changelog", -"section": "๐Ÿ“Š Output Quality", -"url": "about/changelog.html#-output-quality", -"text": "Clean proteome: {sample}-AMAP.clean.faa without pseudogenes or internal stops Annotation QC: per-sample metrics (genome size, GC%, coding density, hypothetical %, annotation rate) GFF3 spec compliance: ##sequence-region pragmas, attribute escaping, sorted features QC report section: interactive table + coding density / hypothetical % / genome size plots" -}, -{ -"page": "Changelog", -"section": "๐Ÿงช Code Quality", -"url": "about/changelog.html#-code-quality", -"text": "58 unit tests: covering IDs, overlaps, reconciliation, validation, clean proteome, QC, merge GitHub Actions CI: Python 3.9 + 3.11 matrix (currently disabled, ready to activate) Type hints: core modules with Coord, Coord3, CoordDict aliases Module docstrings: coordinate conventions, naming schemes documented" -}, -{ -"page": "Changelog", -"section": "๐Ÿ–ฅ๏ธ Usability", -"url": "about/changelog.html#-usability", -"text": "Structured startup banner: shows all configuration in a formatted box --version flag: prints pipeline + Nextflow version Progress logging: โ–ธ [BAKTA] Annotating G26689... for every major step Actionable error messages: boxed errors with cause + fix + re-run command Sample count at startup: confirms TSV was parsed correctly" -}, -{ -"page": "Changelog", -"section": "๐Ÿ“ Documentation", -"url": "about/changelog.html#-documentation-1", -"text": "README: updated for v1.0.0 with all new parameters, outputs, and organism-agnostic usage CHANGELOG: this file Output tree: documents .clean.faa, .qc.tsv, pipeline_info/ new files" -}, -{ -"page": "Changelog", -"section": "๐Ÿ› Bug Fixes (85 total)", -"url": "about/changelog.html#-bug-fixes-85-total", -"text": "20 critical (๐Ÿ”ด) + 65 improvements (๐ŸŸก) See commit history on fix branch for full details" -}, -{ -"page": "Changelog", -"section": "[0.1.0] - 2025", -"url": "about/changelog.html#010---2025", -"text": "Initial release MTBC-focused annotation pipeline Bakta + Miniprot + eggNOG integration HTML report with COG, Mycobrowser, pangenome analysis" -}, -{ -"page": "Citation", -"section": "", -"url": "about/citation.html", -"text": "Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software]. @software{Ruiz-Rodriguez_AMAP_2026, author = {Ruiz-Rodriguez, Paula and Coscolla, Mireia}, doi = {10.5281/zenodo.8024446}, license = {GPL-3.0}, month = apr, title = {{AMAP}}, url = {https://github.com/PathoGenOmics-Lab/AMAP}, version = {1.0.1}, year = {2026}} The repository also carries a CITATION.cff, which GitHub reads to offer a formatted citation from the sidebar." -}, -{ -"page": "Citation", -"section": "Authors", -"url": "about/citation.html#authors", -"text": "Paula Ruiz-Rodriguez and Mireia Coscolla I2SysBio, University of Valencia-CSIC, FISABIO Joint Research Unit Infection and Public Health, Valencia, Spain." -}, -{ -"page": "Citation", -"section": "Cite the tools too", -"url": "about/citation.html#cite-the-tools-too", -"text": "AMAP orchestrates other people's work. If a result depends on one of these, cite it as well. Nextflow Runs the pipeline Bakta Baseline annotation Miniprot Reference proteome alignment eggNOG-mapper Functional annotation Diamond Duplication searches pangene Pangenome graph pathotypr MTBC lineage typing The exact version of each one used by a given run is recorded in results/pipeline_info/software_versions.yml." -}, -{ -"page": "Citation", -"section": "Reference data", -"url": "about/citation.html#reference-data", -"text": "The MTBC reference files bundled in the container come from public repositories. The pathotypr lineage markers are archived on Zenodo under 10.5281/zenodo.19210044, and Mycobrowser functional categories come from mycobrowser.epfl.ch." -}, -{ -"page": "Citation", -"section": "Licence", -"url": "about/citation.html#licence", -"text": "AMAP is released under the GPL-3.0 licence. See LICENSE." -}, -{ -"page": "Contributing", -"section": "", -"url": "about/contributing.html", -"text": "Bug reports, fixes and suggestions are all welcome. The full guidelines live in CONTRIBUTING.md, and the project follows a code of conduct." -}, -{ -"page": "Contributing", -"section": "Reporting a bug", -"url": "about/contributing.html#reporting-a-bug", -"text": "Open an issue at github.com/PathoGenOmics-Lab/AMAP/issues with the command you ran, the failing task's .command.err, and both software_versions.yml and params.yml from pipeline_info/. Those last two pin the exact versions and parameters, which is usually what the answer turns on. Troubleshooting covers the common cases first." -}, -{ -"page": "Contributing", -"section": "Working on the code", -"url": "about/contributing.html#working-on-the-code", -"text": "git clone https://github.com/PathoGenOmics-Lab/AMAP cd AMAP pip install -r requirements-test.txt python -m pytest tests/ -v requirements-test.txt covers everything bin/ imports, not only what the tests import directly, and CI installs from the same file. A shorter list appears to work until a test touches classify_cases.py, which pulls in plotly through classify/plotting.py. The suite covers ID handling, overlap arithmetic, case classification, the rules that resolve and recover genes, and the parsers, and it runs in about a second. Anything touching bin/classify/ should come with a test, sinc" -}, -{ -"page": "Contributing", -"section": "Annotating H37Rv with itself", -"url": "about/contributing.html#annotating-h37rv-with-itself", -"text": "A change to the correction algorithm should also pass the self-test, which has exactly one right answer: annotating H37Rv against its own reference must give back every CDS of the AL123456.3 GenBank record, at exactly its coordinates and strand, under its own name. docker run --rm --platform linux/amd64 -v \"$PWD:/amap\" -w /amap \\ paururo/amap:latest bash tests/selftest/h37rv_selftest.sh GenBank CDS: 4031 AMAP genes: 4031 exact and named right: 4031 not at exactly their coordinates: 0 named wrong: 0 AMAP genes that are no GenBank CDS: 0 The first run spends a couple of minutes on " -}, -{ -"page": "Contributing", -"section": "Pipeline code and -resume", -"url": "about/contributing.html#pipeline-code-and--resume", -"text": "Nextflow hashes a task's inputs and script, not the files the script calls by path, so python3 ${projectDir}/bin/check_cds.py alone would let -resume hand back what the old check_cds.py produced. Every process that runs AMAP code therefore lists those files in ext code, and its script names task.ext.code in a comment, which puts them in the cache key (Nextflow 24.04 and later): ext code: { files(\"${projectDir}/bin/{check_cds.py,deduplicate_rvs.py,classify/*.py}\") } A script that imports classify needs classify/*.py in the list. The reverse also bites: Nextflow adds to a task's cache key any bi" -}, -{ -"page": "Contributing", -"section": "Working on the container", -"url": "about/contributing.html#working-on-the-container", -"text": "docker build --platform linux/amd64 -f docker/Dockerfile -t amap:dev . docker/smoke_test.sh amap:dev The smoke test checks the tools, the Python and R libraries, the bundled reference data, and runs the test suite with the image interpreter. Worth doing before proposing a change to the Dockerfile, since a library update can break the pipeline scripts without the image looking any different." -}, -{ -"page": "Contributing", -"section": "Working on these docs", -"url": "about/contributing.html#working-on-these-docs", -"text": "The website is plain HTML in docs/, with nothing to build on the server: GitHub Pages serves the folder as it is (docs/.nojekyll keeps it from running Jekyll over the markdown), and it opens from a clone too. The long pages are the markdown files, which GitHub also renders, and tools/build_docs.py writes each one's .html next to it, with the site's header, page list, contents and search. python3 tools/build_docs.py # after editing any .md under docs/ python3 tools/check_site.py # what CI runs python3 -m http.server -d docs 8000 # then open http://localhost:8000 The front pag" -}, -{ -"page": "Contributing", -"section": "Style", -"url": "about/contributing.html#style", -"text": "Keep to what is already there: short paragraphs, GitHub's alerts (> [!WARNING]) for the things that actually bite, and concrete numbers rather than \"large\" or \"slow\". Where a threshold appears in the docs it should match the constant in the code, so if you change one, change the other; check_site.py holds the ones it knows to the code." -} -]; +window.AMAP_DOCS_INDEX = (function (d) { + return d.rows.map(function (r) { + var p = d.pages[r[0]]; + var e = { page: p[0], section: r[1], url: p[1] + (r[2] ? "#" + r[2] : ""), text: r[3] }; + if (r[5]) e.code = r[5]; + e.kind = d.kinds[r[4]]; + return e; + }); +})({"pages":[["Installation","getting-started/installation.html"],["Quickstart","getting-started/quickstart.html"],["Tutorial","getting-started/tutorial.html"],["Preparing inputs","getting-started/inputs.html"],["Parameters","user-guide/parameters.html"],["Running AMAP","user-guide/running.html"],["Outputs","user-guide/outputs.html"],["The full report","user-guide/report.html"],["The report by question","user-guide/amap-report.html"],["Custom reference","user-guide/custom-reference.html"],["Overview","how-it-works/index.html"],["The workflow","how-it-works/workflow.html"],["The correction algorithm","how-it-works/correction-algorithm.html"],["Duplication metrics","how-it-works/duplications.html"],["Containers","deployment/containers.html"],["HPC clusters","deployment/hpc.html"],["Troubleshooting","deployment/troubleshooting.html"],["Changelog","about/changelog.html"],["Citation","about/citation.html"],["Contributing","about/contributing.html"],["Front page","index.html"],["Step by step, illustrated","how-it-works.html"]],"kinds":["page","section","option","changelog","faq","story"],"rows":[[0,"","","AMAP needs two things on your machine: Nextflow, which runs the pipeline, and a container engine, which supplies every tool the pipeline calls. You do not install Bakta, miniprot, eggNOG-mapper or any of the rest yourself; they all live inside the AMAP image. You also do not need to clone this repository. Nextflow pulls the pipeline straight from GitHub.",0],[0,"1. Nextflow","1-nextflow","Nextflow needs Java 17 or newer, and Conda brings both: If you do not have Conda, Miniforge is the one to install: it comes set up for conda-forge, the channel Bioconda builds on. Any other Nextflow installation works equally well, see the Nextflow documentation. AMAP requires Nextflow 24.04 or newer, which is enforced by the pipeline manifest.",1,"conda create -n nextflow-env -c conda-forge bioconda nextflow activate"],[0,"2. A container engine","2-a-container-engine","Pick whichever your machine allows. Docker Best on a laptop or workstation where you have root. Follow the Docker installation guide for your platform. Singularity / Apptainer The usual choice on an HPC cluster, where Docker is normally not permitted. Follow the Apptainer installation guide. Many clusters already provide it as a module:",1,"module load singularity or: apptainer"],[0,"3. Check the installation","3-check-the-installation","On a machine with Docker Desktop running, the second line prints something like this: Unlike docker --version, it fails when the Docker daemon is not running, and it prints the memory your containers can really have. Under 8 GB? Raise it in Docker Desktop under Settings, Resources, Memory. With Singularity or Apptainer, singularity --version is the check.",1,"nextflow -version docker info --format 'Docker can give containers {{.NCPU}} CPUs and {{.MemTotal}} bytes' Docker 14 8217014272 bytes"],[0,"4. Run the test profile","4-run-the-test-profile","Before pointing AMAP at real data, confirm the whole chain works. The test profile annotates the H37Rv genome bundled in the image, so it needs no data of your own, and --skip_functional_annotation keeps it to the correction: It downloads nothing beyond the pipeline and its image, writes amap-test/ and Nextflow's work/ in the folder you run it from, and ends with a โœ“ COMPLETED block. On an Apple M4 Pro, under emulation and with the image already pulled, it took 2 min 47 s. The first run downloads the AMAP image, about 3.4 GB compressed and 12 GB on disk, because it bundles the Bakta database and the H37Rv reference files. Expect that to take a while. Later runs reuse the cached image.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker,test --skip_functional_annotation --outdir amap-test"],[0,"The eggNOG database","the-eggnog-database","Without --skip_functional_annotation, the test also builds the eggNOG database, the one thing the image does not hold. It is the largest thing AMAP keeps: eggnog.db alone unpacks to 41.4 GB from a 6.8 GB download, and the build also downloads the 9.5 GB eggNOG proteome, makes the bacteria-only Diamond index from it and deletes it. It is built once, in the folder --eggnog_db names, and every run after reuses it, so point that somewhere with room. The annotation then loads it into memory and asks for 50 GB, where the test profile caps every task at 8 GB and 1 h, so keep --skip_functional_annotation for the test. Check inputs before a long run Once you have your own samples, --validate_only checks every FASTA and database path without executing anything: See inputs for what it looks for.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --validate_only"],[0,"Running without a container","running-without-a-container","Possible but not recommended: you take on installing Bakta, miniprot, eggNOG-mapper, Diamond, pangene, pathotypr, R and the report packages yourself, and pointing AMAP at each reference file with the reference arguments. The repository ships an environment.yml that reproduces the software stack under Conda: You will still need to supply the Bakta database and the reference files, which the container would otherwise provide. See containers for what the image contains.",1,"conda env create -f environment.yml activate amap"],[0,"Next steps","next-steps","Quickstart: your first real run Preparing inputs: the sample sheet and what makes a genome pass validation",1],[0,"Check inputs before a long run","note-check-inputs-before-a-long-run","Once you have your own samples, --validate_only checks every FASTA and database path without executing anything:",1],[1,"","","This walks from an installed Nextflow to an open report. If Nextflow or a container engine is still missing, start at installation.",0],[1,"1. Check the plumbing","1-check-the-plumbing","The test profile annotates the H37Rv genome that ships inside the image, so it needs no data of your own, and --skip_functional_annotation keeps it to the correction, with no eggNOG database to build: It writes amap-test/ and work/ where you run it, and ends with a โœ“ COMPLETED block. On an Apple M4 Pro, under emulation and with the image already pulled, it took 2 min 47 s; the first run also pulls the image, about 3.4 GB compressed and 12 GB on disk, once.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker,test --skip_functional_annotation --outdir amap-test"],[1,"2. Write a sample sheet","2-write-a-sample-sheet","AMAP reads a tab-separated file with two columns, sampleId and fasta: The separator must be a real tab. Paths can be absolute, or relative to where you launch Nextflow. See preparing inputs for the details and the checks AMAP runs over them.",1,"sampleId fasta G26689 /data/genomes/G26689.fasta G26690 /data/genomes/G26690.fasta"],[1,"3. Check the inputs","3-check-the-inputs","Cheap, and it catches the mistakes that would otherwise surface twenty minutes into a run: When the check passes, its verdict and any warnings are in results/pipeline_info/validation.log, and the terminal says only that the step ran. When it fails, the run stops and the terminal prints the whole report.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --validate_only"],[1,"4. Run it","4-run-it","Pick what you have, and copy the command: Or start from one of these: Docker Singularity SLURM cluster Without an executor profile every task is capped at the machine's CPUs and 16 GB, and with the executor profiles at 48 CPUs and 128 GB, whichever order the profiles come in; a larger request is lowered to the cap. Nextflow refuses to start a task asking for more memory than the machine has, so on a smaller machine, lower the ceiling to what docker info says your containers can have, and leave eggNOG out: it loads its whole database into memory, about 45 GB, whatever the ceiling. The tutorial has the per-step figures and what to do when the machine cannot reach them.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv singularity singularity,slurm --max_cpus 4 --max_memory 8.GB --skip_functional_annotation"],[1,"5. Read the results","5-read-the-results","Everything lands under results/ unless you set --outdir. Each sample's folder is named {lineage}_{sampleId}: lineage typing puts the lineage in front, so G26689 comes out as L4_G26689, and under --skip_typing it is the sample ID alone (outputs has the details). The things to open first: Each genome and each gene, question by question, in every run: results/amap_report.html. The full report, after a run with eggNOG: results/final_report.html. Corrected annotation, per sample, after a run with eggNOG: results/AMAP/{lineage}_{sampleId}/*-AMAP.gff3. Protein sequences for downstream tools, after a run with eggNOG: results/AMAP/{lineage}_{sampleId}/*-AMAP.clean.faa. The corrected genes, their proteins and sequences, in every run: results/AMAP/{lineage}_{sampleId}/*.AMAP.tsv, .faa, .fna. Which protein file do I want? *-AMAP.faa holds every gene, pseudogenes included, so it mirrors the annotation exactly. *-AMAP.clean.faa drops pseudogenes and sequences with internal stop codons, which is what orthology, phylogenomics and dN/dS tools expect. Reach for the clean one unless you specifically need the complete set. The outputs page covers every directory.",1],[1,"6. Resume an interrupted run","6-resume-an-interrupted-run","Nextflow caches completed tasks. If a run dies, restart it with -resume and it picks up where it stopped: Keep the same parameters. Changing an input invalidates the cache for every task downstream of it, which is what you want, but it does mean the run redoes that work.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv -resume"],[1,"Common variations","common-variations","An organism outside the Mycobacterium tuberculosis complex (MTBC). Lineage typing and Mycobrowser are specific to the complex, so turn them off and supply your own reference: See custom reference. Just the corrected genes. If you do not need functional annotation, duplications or the pangenome, stop after the correction step. It is much faster, and leaves the corrected genes as a table with their proteins and sequences; the GFF3 and GenBank files come after eggNOG, so there are none. There is no final_report.html; amap_report.html is still written, without its eggNOG sections:",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --skip_typing --skip_mycobrowser --genus Escherichia --species \"Escherichia coli\" --ref_annotation my_reference.gbk --ref_genome my_reference.fasta --skip_functional_annotation"],[1,"Next steps","next-steps","Running AMAP: profiles, schedulers and larger runs Parameters: the full reference Overview: what the correction actually does",1],[1,"Which protein file do I want?","note-which-protein-file-do-i-want","*-AMAP.faa holds every gene, pseudogenes included, so it mirrors the annotation exactly.",1],[2,"","","Every command spelled out, nothing assumed. It uses a real genome you can download in one line, so you can follow along and compare your numbers against the ones printed here. They come from actual runs of these commands, made on 18 October 2026 with AMAP at commit 947a99d. For exactly these numbers, add this to each nextflow run: Pinning the pipeline says why the whole hash. The runs skipped eggNOG, with --skip_functional_annotation, and no number here depends on it. The one table eggNOG's step writes, in the sample check, was made by hand from the same run. The genome is Mycobacterium tuberculosis CDC1551, corrected against the H37Rv reference that ships inside the AMAP container. At the end there is a second pass showing the same thing for an organism outside the Mycobacterium tuberculosis complex (MTBC). The correction itself takes minutes on one genome: 2 min 16 s here, without eggNOG. What dominates the first full run is setup: pulling the image, 3.4 GB compressed and 12 GB on disk, and building the eggNOG database, both of which happen once and are reused afterwards. Expect the first run to take considerably longer than every run after it, and do not read a long silence during the database build as a failure.",0,"-r 947a99dee49d2a2429e014296ac35ba08e911108"],[2,"1. What you need","1-what-you-need","Nextflow and a container engine. Every tool the pipeline itself calls lives inside the AMAP image, so there is nothing else to install. The easiest way to get Nextflow is Conda. If you do not have it, Miniforge is the one to install: Nextflow needs Java 17 or newer. Check what you have: Nextflow needs Java 17 or newer On Java 8 or 11, Nextflow stops with \"Cannot find Java or it's a wrong version\". Install a newer JDK, for example conda install -c conda-forge openjdk=17. Then a container engine. Docker on a laptop, Singularity or Apptainer on a cluster. With Docker, ask it what your containers can have: That is the machine behind this page: an Apple M4 Pro, with Docker Desktop given 8 GB of its 24. The command fails when the Docker daemon is not running, which docker --version does not catch. Under 8 GB? Raise it in Docker Desktop under Settings, Resources, Memory. On a cluster, singularity --version. Full details, including running without a container, are on the installation page. On an Apple Silicon Mac, read this first The AMAP image is linux/amd64 only, so on an M-series Mac every step runs under emulation. The test profile in step 2 is fine. A real genome is slow enough that a working run can look like a hung one, which is the part worth knowing before you start wondering what you broke. It is a reasonable way to learn what the pipeline does and to check your sample sheet. For results you intend to use, move to an amd64 machine or a cluster.",1,"conda create -n nextflow-env -c conda-forge bioconda nextflow activate java -version docker info --format 'Docker can give containers {{.NCPU}} CPUs and {{.MemTotal}} bytes' Docker 14 8217014272 bytes"],[2,"The machine","the-machine","Worth checking before you start, because this is what stops a run rather than anything you will have got wrong. Disk goes first. The container is 12 GB unpacked. The eggNOG database is built on first use and cached, and it is the largest thing a run keeps: eggnog.db alone unpacks to 41.4 GB from a 6.8 GB download. For its bacteria-only Diamond index AMAP also downloads the 9.5 GB eggNOG proteome, and deletes it once the index is built. Nextflow keeps every intermediate under work/, which came to 97 MB for the test in step 2 and 248 MB for all the runs of this page, none with eggNOG, and grows with the sample sheet. Memory decides whether the run finishes at all. Two steps are heavy: eggNOG annotation: 24, 50 GB. eggNOG database build: 16, 45 GB. Pangenome: 16, 16 GB. Duplication analysis: 6 to 13, 10 GB. Report: 1, 8 GB. Bakta annotation: 12, 6 GB. Everything else: 1 to 8, 6 GB or less. Nextflow will not start a task that asks for more memory than the machine has, so on a laptop these requests are the thing that fails, not your data. Lower the ceiling to what docker info reports, and every request is clamped to it. Leave eggNOG out as well: it loads its whole database into memory, about 45 GB, whatever the ceiling, so no cap makes it fit. You still get the corrected genes, which is what AMAP exists to produce, as a table with their proteins and sequences, and amap_report.html. What you lose is eggNOG categories, the GFF3 and GenBank files written after it, the pangenome, the duplication analysis and final_report.html. The heaviest step left asks for 6 GB. Docker Desktop has its own memory limit, and it is the one that counts On macOS and Windows, Docker runs your containers inside a virtual machine with a memory ceiling of its own, often 8 GB by default no matter how much the machine has. Nextflow sizes its requests against the host, so it will happily start a 12 GB task that the VM then kills. You get exit code 137 and nothing that mentions Docker. Raising --max_memory makes this worse, not better. Raise the VM instead, in Docker Desktop under Settings, Resources, Memory, and keep --max_memory at or below what you set there. Linux has no such layer. Containers use host memory directly and this does not apply.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --max_cpus 4 --max_memory 8.GB --skip_functional_annotation"],[2,"2. Check the plumbing before touching real data","2-check-the-plumbing-before-touching-real-data","The test profile annotates the H37Rv genome that ships inside the container, so it needs no data of your own, and --skip_functional_annotation keeps it to the correction, with no eggNOG database to build: The first run pulls the AMAP image. It is around 3.4 GB compressed and 12 GB on disk, because it carries the Bakta database and the H37Rv reference files. That is a one-off; later runs reuse it. With the image already pulled, the test took 2 min 47 s on the machine above, and wrote amap-test/ and work/ in the folder it ran from. If the whole chain works, it ends like this, and then names the work directory and the command:",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker,test --skip_functional_annotation --outdir amap-test COMPLETED Pipeline v1.0.1 Nextflow 26.10.2 Profile Duration 2m 47s Succeeded 10 Failed 0 Ignored Output"],[2,"3. Get a genome","3-get-a-genome","Make a working directory and fetch an assembly: Have a look at what you got: One closed contig, 4,403,837 bp. AMAP takes assembled genomes, so reads are not an input here.",1,"mkdir amap-tutorial cd NCBI=https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF ASM=000/008/585/GCF_000008585.1_ASM858v1/GCF_000008585.1_ASM858v1_genomic curl -sL -o cdc1551.fna.gz \"$NCBI/$ASM.fna.gz\" gunzip grep cdc1551.fna >NC_002755.2 Mycobacterium tuberculosis CDC1551, complete sequence"],[2,"4. Write the sample sheet","4-write-the-sample-sheet","A tab-separated file with exactly two columns: It has to be a real tab Spaces will not do, and this is the single most common reason a sample sheet is rejected. Check with cat -t samples.tsv: real tabs show up as ^I. (cat -A does the same on Linux but is not accepted by the cat that ships with macOS.) Keep sample IDs to letters, digits and underscores. The classification cuts names at the first dot, so a sample G26689.1 would be set aside and missing from the results.",1,"printf 'sampleId\\tfasta\\nCDC1551\\t%s/cdc1551.fna\\n' \"$PWD\" samples.tsv cat sampleId fasta CDC1551 /path/to/amap-tutorial/cdc1551.fna"],[2,"5. Validate before committing to a long run","5-validate-before-committing-to-a-long-run","Checks that pass write their verdict to results/pipeline_info/validation.log; one that fails stops the run and prints the whole report instead. The verdict is two lines: The log's last line then starts โœ“ PASSED. This costs seconds and catches the mistakes that would otherwise surface minutes in. Preparing inputs lists every check and its threshold.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --validate_only grep -E '^(Results|Total)' results/pipeline_info/validation.log Results: 1 OK, 0 warnings, failed Total: error(s), warning(s)"],[2,"6. Run it","6-run-it","What happens, in order: lineage typing, Bakta annotation, alignment of the H37Rv proteome onto your genome, case classification, correction, eggNOG functional annotation, duplication analysis, pangenome, the two reports. The first run also downloads and builds the eggNOG database, eggnog.db alone 41.4 GB, and that dominates the wall clock. Point --eggnog_db at somewhere persistent so the next run reuses it: If the run dies partway, -resume picks up from the last completed step.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --eggnog_db ~/databases/eggnog"],[2,"How long it took","how-long-it-took","Without eggNOG, on the machine above, the CDC1551 run took 2 min 16 s from start to finish. Each step, from the run's execution trace in results/pipeline_info/, its realtime and peak_rss: PATHOTYPR, lineage typing: 1.1 s, 6.1 MB. BAKTA_ANNOTATOR: 1 min 46 s, 3.5 GB. MINIPROT: 2 s, 324 MB. TYPE_CASES, the classification: 14.2 s, 219.2 MB. FIX_CDS, the correction: 3.7 s, 165.2 MB. AMAP_REPORT: 944 ms, 35.6 MB. The run's other four tasks, which set up the reference and record the parameters, the versions and the validation, took under 4 s each. Bakta is the step to plan for: the E. coli run of step 9 took 5 min 47 s, of which Bakta on Sakai took 2 min 28 s at 5.7 GB, and PREPARE_REFERENCE, which runs Bakta on the reference itself, 2 min 36 s.",1],[2,"7. Look at what came out","7-look-at-what-came-out","That is a run without eggNOG, like the ones behind this page. A full run adds DUPLICATIONS/, PANGENE/ and final_report.html. Open results/amap_report.html first: every run writes it, and it goes through the run question by question. After a full run, results/final_report.html has the whole analysis. Both are self-contained, so they travel by email.",1,"ls -1F results/ AMAP/ BAKTA/ CDS_CASES/ PATHOTYPR/ amap_report.html pipeline_info/"],[2,"Did the correction do anything?","did-the-correction-do-anything","Every file of the sample carries its name as lineage typing leaves it, L4_CDC1551. The case summary's first two columns are the counts: The other three, Sequence, Lineage and Sample, say whose counts they are, which matters once the folder holds several samples. What each row means: case2: 2,813, Coordinates matched the reference exactly. case3: 24, Coordinates matched, the alignment's length differed. case4: 1,100, Overlapped the reference but disagreed, and was resolved. case1_initial: 141, Bakta called it, the reference has nothing there. rv_missing: 97, The reference has a gene there, Bakta missed it. total_cds: 4,077, What Bakta called before the correction. case1_removed: 130, \"Novel\" genes that were not really novel. case1_kept: 11, Kept as genuinely absent from H37Rv. Read that as: of 4,077 Bakta calls, 2,813 already agreed with the reference, 1,100 had to be adjudicated, and 97 reference genes Bakta had missed were rescued. Only 11 of the 141 apparently novel genes survived scrutiny, which is what you would expect for a strain of the same species as the reference. Cases 2, 3, 4 and the initial case 1 add up to 4,078, one more than the calls. One call, L4_CDC1551_03643, has the same ends as two reference proteins: Rv3478 at its own length, which makes it case 2, and Rv1361c, a PPE protein three residues longer, which makes it case 3 too.",1,"cut -f1,2 results/CDS_CASES/summary/L4_CDC1551-summary.tsv column -t Type Number case2 2813 case3 24 case4 1100 case1_initial 141 rv_missing 97 total_cds 4077 case1_removed 130 case1_kept 11"],[2,"Sanity-check the sample","sanity-check-the-sample","Six of its eleven columns: 65.6% GC and 91.3% coding density are what a healthy MTBC genome looks like. A number well outside that is worth investigating before trusting the rest. This table is written after eggNOG, so a run without it has none. The line above was made by running that step's own script, GET_OUTPUTS, on the corrected genes of the run behind this page, with an eggNOG table that has no rows. None of the six columns depends on eggNOG; the one that does, eggnog_annotation_pct, is left out.",1,"column -t -s$'\\t' results/AMAP/*/*-AMAP.qc.tsv genome_size n_contigs gc_pct total_cds pseudogene_count coding_density_pct 4403837 1 65.61 4014 123 91.3"],[2,"The files you will actually use","the-files-you-will-actually-use","Lineage typing puts the lineage in front of the sample ID, so CDC1551 comes out as L4_CDC1551: Corrected annotation: results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gff3, after eggNOG. Same, GenBank: results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gbk, after eggNOG. Proteins for downstream tools: results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.clean.faa, after eggNOG. Everything known per gene: results/AMAP/L4_CDC1551/L4_CDC1551-AMAPmastertable.tsv, after eggNOG. Tool versions for your methods: results/pipeline_info/software_versions.yml, every run. The run above skipped eggNOG, so in their place it has L4_CDC1551.AMAP.tsv, L4_CDC1551.AMAP.faa and L4_CDC1551.AMAP.fna, with a dot: the corrected genes, their proteins and their sequences. Two protein files, and they differ on purpose *-AMAP.faa has one entry per annotated gene, pseudogenes included, so it matches the GFF3 gene for gene. *-AMAP.clean.faa drops pseudogenes and anything with an internal stop codon, which is what orthology, phylogenomics and dN/dS tools expect. It is legitimately shorter. Every directory is documented under outputs.",1],[2,"8. More than one genome","8-more-than-one-genome","Add rows. The pipeline parallelises across samples, and the pangenome and the cross-genome duplication analysis only become meaningful with several: Past about twenty genomes, move to a cluster: HPC clusters covers scratch, shared filesystems and where things should live.",1,"sampleId fasta CDC1551 /data/cdc1551.fna Erdman /data/erdman.fna Beijing_1 /data/beijing_1.fna nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv --eggnog_db /shared/databases/eggnog"],[2,"9. The same thing for another organism","9-the-same-thing-for-another-organism","Everything above assumed MTBC. For anything else you supply the trusted reference yourself, and turn off the two MTBC-specific steps. Fetch a reference and a sample. Here, E. coli O157:H7 Sakai corrected against K-12 MG1655: --skip_typing because lineage typing is specific to tuberculosis, and --skip_mycobrowser because those functional categories are an MTBC resource. Before annotating anything, AMAP extracts the proteome from your reference, annotates that same reference genome with Bakta, and takes the difference to work out which genes Bakta invents. For MTBC that list was curated by hand; here it is derived automatically. On this pair it found 142. The case counts come out differently from the MTBC run, and the difference is biology rather than a problem. Without lineage typing the sample keeps its name: case1_kept counts the genes O157:H7 has and K-12 does not. Those 1,387 are the prophages and the LEE island that O157:H7 carries and K-12 lacks. Against a reference of your own species you should expect a large case 1 when the strains genuinely differ, and a small one when they do not. The annotation and the genome must be the same assembly Coordinates from the annotation are used directly against the FASTA. An annotation from a different assembly produces nonsense: proteins that do not translate, and a nearly empty reference proteome. AMAP writes how many proteins it extracted into the work directory, where grep -h 'Curated proteins' work///.command.out finds it (4,318 on this pair), so check the number looks like a genome's worth. Custom reference goes into what makes a good reference and what changes when you use one.",1,"NCBI=https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF K12=000/005/845/GCF_000005845.2_ASM584v2/GCF_000005845.2_ASM584v2_genomic SAKAI=000/008/865/GCF_000008865.2_ASM886v2/GCF_000008865.2_ASM886v2_genomic the reference: annotation plus genome it describes curl -sL -o ref.gbk.gz \"$NCBI/$K12.gbff.gz\" ref.fna.gz \"$NCBI/$K12.fna.gz\" sample sakai.fna.gz \"$NCBI/$SAKAI.fna.gz\" gunzip printf 'sampleId\\tfasta\\nSakai\\t%s/sakai.fna\\n' \"$PWD\" ecoli.tsv nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv --skip_typing --skip_mycobrowser --genus Escherichia --species \"Escherichia coli\" --ref_annotation ref.gbk --ref_genome ref.fna cut -f1,2 results/CDS_CASES/summary/Sakai-summary.tsv column -t Type Number case2 3324 case3 31 case4 507 case1_initial 1506 rv_missing 133 total_cds 5367 case1_removed 122 case1_kept 1387"],[2,"When something goes wrong","when-something-goes-wrong","Nextflow prints the work directory of the failed task. Everything you need is in it: Troubleshooting is organised by the error you are looking at.",1,"work/ab/cd1234.../ .command.sh the exact script that ran .command.err stderr on its own, almost always answer .exitcode"],[2,"Next steps","next-steps","Parameters: every flag and its default Overview: what the correction actually does, and the thresholds behind it Outputs: every file the run produces",1],[2,"Nextflow needs Java 17 or newer","note-nextflow-needs-java-17-or-newer","On Java 8 or 11, Nextflow stops with \"Cannot find Java or it's a wrong version\".",1],[2,"On an Apple Silicon Mac, read this first","note-on-an-apple-silicon-mac-read-this-first","The AMAP image is linux/amd64 only, so on an M-series Mac every step runs under emulation.",1],[2,"Docker Desktop has its own memory limit, and it is the one that counts","note-docker-desktop-has-its-own-memory-limit-and-it-is-the-one-that-counts","On macOS and Windows, Docker runs your containers inside a virtual machine with a memory ceiling of its own, often 8 GB by default no matter how much the...",1],[2,"It has to be a real tab","note-it-has-to-be-a-real-tab","Spaces will not do, and this is the single most common reason a sample sheet is rejected.",1],[2,"Two protein files, and they differ on purpose","note-two-protein-files-and-they-differ-on-purpose","*-AMAP.faa has one entry per annotated gene, pseudogenes included, so it matches the GFF3 gene for gene.",1],[2,"The annotation and the genome must be the same assembly","note-the-annotation-and-the-genome-must-be-the-same-assembly","Coordinates from the annotation are used directly against the FASTA.",1],[3,"","","AMAP takes assembled genomes. It does not do assembly, so reads are not an input here.",0],[3,"The sample sheet","the-sample-sheet","A tab-separated file with a header row and exactly two columns: sampleId: The name carried through every output directory and file. Must be unique. Lineage typing puts the lineage in front, so G26689 comes out as L4_G26689. fasta: Path to the assembled genome. Absolute, or relative to the directory you launch Nextflow from. The separator has to be an actual tab. Spaces will not do, and this is the most common reason a sample sheet is rejected. Avoid dots in sample IDs The classification cuts names at the first dot, so for G26689.1 it writes its tables as L4_G26689 while the pipeline waits for L4_G26689.1. The step fails, is tried twice more and is set aside, and the run finishes without that sample. Validation does not check this: stick to letters, digits and underscores.",1,"sampleId fasta G26689 /data/genomes/G26689.fasta G26690 /data/genomes/G26690.fasta G12345 ./assemblies/G12345.fa"],[3,"Pre-flight validation","pre-flight-validation","VALIDATE_INPUTS runs before any computation, on every run except -profile test, which has no sample sheet. To run only the checks and stop: A check that passes writes its results to results/pipeline_info/validation.log; one that fails stops the run and prints them. Two mistakes in the sheet itself stop it even earlier, before any task starts, with a message of the pipeline's own and no log at all: the drawing of the sheet above shows both.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --validate_only"],[3,"What fails the run","what-fails-the-run","Genome size: below 100 kb, or above 20 Mb. N content: above 10%. GC content: below 20% or above 80%. Empty sequences: any record with no bases. Duplicate contig headers: any repeated identifier. Sample sheet: missing sampleId or fasta column, no data rows, duplicate or empty sample IDs. FASTA path: file does not exist or cannot be parsed. Bakta database: not found, or no version.json or bakta.db in it. eggNOG database: eggnog_db.ok in it while the database is incomplete, since Nextflow then never builds it again. Reference files: --ref_annotation, --ref_genome or a reference argument that the run reads but that is neither on the host nor inside the container.",1],[3,"What only warns","what-only-warns","Genome size: below 500 kb, or above 10 Mb. N content: above 1%. GC content: below 25% or above 75%. Assembly fragmentation: more than 100 contigs, and again above 500. Short contigs: any contig under 200 bp in a multi-contig assembly. Non-standard characters: anything outside ATGCNRYSWKMBDHV. eggNOG database: a bacteria.dmnd under 100 MB, which EGGNOG_DB takes for a truncated download and builds again. Mycobrowser table: --mycobrowser_ref neither on the host nor inside the container, so the master table goes without Mycobrowser categories. Warnings do not stop a run. Add --strict_validation to have them stop it too. An eggNOG database that does not exist yet is neither: the first run that annotates builds it at --eggnog_db. The thresholds are deliberately wide, since AMAP is not restricted to tuberculosis. A 4.4 Mb genome of the Mycobacterium tuberculosis complex (MTBC) at 65% GC sits comfortably inside them, and so does most of the bacterial world.",1],[3,"Assembly quality","assembly-quality","The correction compares your annotation against a reference proteome, so assembly quality propagates directly into the result. Contiguity matters. A gene that a contig end cuts comes back in pieces, each under the gene's name and marked partial at its open end, the way NCBI marks it. A piece under 8 residues is too short to place: in CDC1551 cut into 40 contigs, 22 of the 25 smaller pieces were placed, and the other three were five or six codons long. The more fragmented the assembly, the more partial genes; pieces of cut genes has the rule. Ns break coding sequences. A run of Ns inside a gene stops it translating, so it is dropped or flagged as a pseudogene. The 1% warning is worth taking seriously. Contig names are kept AMAP runs Bakta with --keep-contig-headers, so the sequence names you supplied are the ones in BAKTA/ and in the corrected annotation, and the corrected tables say which contig each gene is on. Without that option Bakta relabels everything contig_1, contig_2 and so on, and classify_cases.py stops with an error on a multi-contig assembly rather than compare genes across contigs. Closed genomes are read as circular from their header Bakta takes a sequence for a complete, circular replicon when the description after its name says so: complete, as NCBI's \"complete genome\" and \"complete sequence\" do, or circular=true, as Unicycler's does. It then joins the two ends, and a gene across the origin comes out whole. Lineage typing renames a genome of one sequence after its sample and keeps that description. A closed genome whose header says neither is annotated as linear, and a gene across its origin is cut in two there; add circular=true after its name to have it read as circular. Draft assemblies Every contig of a draft assembly is numbered from 1, so the same coordinates turn up on several contigs. The correction keys every gene on its contig as well as its start, end and strand, only ever compares genes on the same contig, and translates each one from its own contig. Cut into 40 contigs, CDC1551 gets every gene that lies within one contig exactly as the closed genome does.",1],[3,"Reference data","reference-data","For MTBC you need nothing else. The container bundles the H37Rv proteome, the Bakta light database, the Mycobrowser table and the pathotypr lineage markers. See containers for the full list. For any other organism, supply your own reference with --ref_annotation and --ref_genome. See custom reference.",1],[3,"Next steps","next-steps","Quickstart: run it Parameters: the full flag reference",1],[3,"Avoid dots in sample IDs","note-avoid-dots-in-sample-ids","The classification cuts names at the first dot, so for G26689.1 it writes its tables as L4_G26689 while the pipeline waits for L4_G26689.1.",1],[3,"Contig names are kept","note-contig-names-are-kept","AMAP runs Bakta with --keep-contig-headers, so the sequence names you supplied are the ones in BAKTA/ and in the corrected annotation, and the corrected tables...",1],[3,"Closed genomes are read as circular from their header","note-closed-genomes-are-read-as-circular-from-their-header","Bakta takes a sequence for a complete, circular replicon when the description after its name says so:",1],[3,"Draft assemblies","note-draft-assemblies","Every contig of a draft assembly is numbered from 1, so the same coordinates turn up on several contigs.",1],[4,"","","Every AMAP parameter, what it defaults to, and when you would change it. Run nextflow run PathoGenOmics-Lab/AMAP --help for the same list at the terminal. Pipeline parameters take two dashes (--tsv). Nextflow's own options take one (-profile, -resume).",0],[4,"Input and output","input-and-output","",1],[4,"Skip options","skip-options","Their effect on the output tree is covered under outputs.",1],[4,"Annotation","annotation","Bakta uses these to pick naming conventions and reference sets, so setting them correctly improves the baseline annotation that AMAP then corrects.",1],[4,"Resources","resources","These are ceilings, not requests. Each process asks for what it needs and is then clamped to these values, so lowering them makes AMAP fit a smaller machine rather than making it run faster. The defaults are a single machine's. The executor profiles raise them to 48 CPUs and 128 GB, -profile highmem to 48 CPUs, 256 GB and 120 h, and -profile test sets 4 CPUs, 8 GB and 1 h, in whatever order -profile names them. Memory scales on retry: a process killed by the out-of-memory killer is resubmitted with more, up to the ceiling.",1],[4,"Validation and information","validation-and-information","",1],[4,"Custom reference","custom-reference","Supply both to annotate any prokaryote against your own trusted annotation. They must be given together; setting only one is an error. See custom reference for what AMAP does with them.",1],[4,"Reference and database arguments","reference-and-database-arguments","You almost certainly do not need these Every file below ships inside the Docker and Singularity images, under /reference/. Running with -profile docker or -profile singularity picks them up automatically. They exist for running without a container, for swapping in updated reference files, or for adapting the pipeline by hand. A database or a file of your own on the host, the full Bakta database or a newer Mycobrowser table for instance, is mounted into the tasks that use it, so it works with the container profiles too. A path that is not on the host is read inside the container, where the defaults are, and validation stops a run whose file is in neither place. The eggNOG database is the one exception: it is not bundled, because of its size. AMAP downloads and builds it on the first run and caches it at --eggnog_db, so point that at somewhere persistent and shared if you run on a cluster.",1],[4,"Nextflow options worth knowing","nextflow-options-worth-knowing","These belong to Nextflow rather than AMAP, so they take a single dash. -profile: Selects execution profiles, comma separated: docker, singularity, standard, slurm, sge, pbs, lsf, test, highmem. -resume: Reuse cached results from the previous run. -r: Run a given branch, tag or full commit hash of AMAP; see pinning the pipeline. -work-dir: Where intermediate files go. Defaults to ./work, and it gets large. -process.queue: Override the scheduler queue or partition. -process.clusterOptions: Extra flags passed to the scheduler, such as '--account=mygroup'. See running AMAP for how the profiles combine.",1],[4,"You almost certainly do not need these","note-you-almost-certainly-do-not-need-these","Every file below ships inside the Docker and Singularity images, under /reference/.",1],[4,"--tsv","opt-tsv","Sample sheet with sampleId and fasta columns, tab separated. Type: file. Default: samples.tsv. Required: Yes, except with -profile test. In Input and output.",2],[4,"--outdir","opt-outdir","Directory for results. Type: string. Default: results. Required: No. In Input and output.",2],[4,"--skip_typing","opt-skip_typing","Skip pathotypr lineage typing. Required outside the Mycobacterium tuberculosis complex (MTBC). Type: boolean. Default: false. In Skip options.",2],[4,"--skip_functional_annotation","opt-skip_functional_annotation","Stop after the correction: no eggNOG, duplications, pangenome or final_report.html; amap_report.html is still written, without its eggNOG sections. Type: boolean. Default: false. In Skip options.",2],[4,"--skip_mycobrowser","opt-skip_mycobrowser","Skip Mycobrowser functional categories. Auto-enabled when --genus is not Mycobacterium and a custom reference is set. Type: boolean. Default: false. In Skip options.",2],[4,"--genus","opt-genus","Genus passed to Bakta. Type: string. Default: \"Mycobacterium\". In Annotation.",2],[4,"--species","opt-species","Species passed to Bakta. Type: string. Default: \"Mycobacterium tuberculosis\". In Annotation.",2],[4,"--max_cpus","opt-max_cpus","Ceiling on CPUs for any single process. Type: integer. Default: the machine's CPUs. In Resources.",2],[4,"--max_memory","opt-max_memory","Ceiling on memory for any single process. Type: string. Default: 16.GB. In Resources.",2],[4,"--max_time","opt-max_time","Ceiling on wall-clock time for any single process. Type: string. Default: 72.h. In Resources.",2],[4,"--scratch","opt-scratch","Use node-local scratch for I/O-heavy processes. On by default in the executor profiles. Type: boolean. Default: false. In Resources.",2],[4,"--validate_only","opt-validate_only","Run input validation and stop. Type: boolean. Default: false. In Validation and information.",2],[4,"--strict_validation","opt-strict_validation","Stop on validation warnings too, not only on errors. Type: boolean. Default: false. In Validation and information.",2],[4,"--help","opt-help","Print the help text and exit. Type: boolean. Default: false. In Validation and information.",2],[4,"--version","opt-version","Print pipeline and Nextflow versions and exit. Type: boolean. Default: false. In Validation and information.",2],[4,"--ref_annotation","opt-ref_annotation","Trusted reference annotation, GFF3 or GenBank. Type: file. Default: null. In Custom reference.",2],[4,"--ref_genome","opt-ref_genome","Trusted reference genome, FASTA. Type: file. Default: null. In Custom reference.",2],[4,"--bakta_db","opt-bakta_db","Bakta database directory, inside the container or on the host. Default inside the container: /data/bakta_db/db-light. In Reference and database arguments.",2],[4,"--eggnog_db","opt-eggnog_db","Pre-formatted eggNOG database directory. Default inside the container: eggnog_db. In Reference and database arguments.",2],[4,"--pathotyper_snps","opt-pathotyper_snps","Lineage marker SNPs for pathotypr. Default inside the container: /reference/snps_lineages.tsv. In Reference and database arguments.",2],[4,"--pathotyper_ref","opt-pathotyper_ref","Reference FASTA for pathotypr. Default inside the container: /reference/MTB_ancestor_reference.fasta. In Reference and database arguments.",2],[4,"--miniprot_ref_aa","opt-miniprot_ref_aa","Reference proteome aligned onto each genome. Default inside the container: /reference/aa_H37Rv.translated.fas. In Reference and database arguments.",2],[4,"--miniprot_ref_bad_bakta","opt-miniprot_ref_bad_bakta","Known problematic annotations, used to filter novel-gene candidates. Default inside the container: /reference/bad.bakta.translated.fas. In Reference and database arguments.",2],[4,"--miniprot_ref_h37rv","opt-miniprot_ref_h37rv","Reference genome FASTA. Default inside the container: /reference/new_refH37Rv.fasta. In Reference and database arguments.",2],[4,"--bakta_proteins","opt-bakta_proteins","Reference proteins GenBank file for Bakta. Default inside the container: /reference/sequence.gb. In Reference and database arguments.",2],[4,"--mycobrowser_ref","opt-mycobrowser_ref","Mycobrowser functional categories table. Default inside the container: /reference/Mycobacterium_tuberculosis_H37Rv_txt_v4.txt. In Reference and database arguments.",2],[5,"","","Nextflow runs the pipeline straight from GitHub, so there is nothing to clone and nothing to keep up to date by hand.",0,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv"],[5,"Profiles","profiles","A profile is a bundle of settings. You normally combine one container profile with one executor profile. docker: container, Runs every process in the AMAP Docker image. singularity: container, Same image via Singularity or Apptainer. standard: container, No container. Every tool must already be on PATH. slurm: executor, Submits each process as a SLURM job. sge: executor, Submits to Sun Grid Engine or Univa Grid Engine. pbs: executor, Submits to PBS or Torque. lsf: executor, Submits to IBM Spectrum LSF. test: data, Small bundled dataset, for checking the installation. highmem: resources, Raises the ceilings to 48 CPUs, 256 GB, 120 h. Combine them with commas: The order does not matter: each profile sets its own kind of thing. The ceilings on what any one task may ask for come from which profiles are on, not from their order: 48 CPUs, 256 GB and 120 h with highmem, 4 CPUs, 8 GB and 1 h with test, 48 CPUs, 128 GB and 72 h with an executor profile, and otherwise the machine's CPUs, 16 GB and 72 h. Without an executor profile everything runs on the local machine.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv"],[5,"On a laptop or workstation","on-a-laptop-or-workstation","CPUs are auto-detected and memory is capped at 16 GB. On a laptop under Docker Desktop, the VM's limit, often 8 GB, is the one that counts: lower the cap to what docker info reports, and skip eggNOG, which asks for 50 GB: On a workstation with more, raise it: What each step needs. A handful of genomes is fine locally. Past about twenty genomes, a cluster is worth the setup.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --max_cpus 4 --max_memory 8.GB --skip_functional_annotation 16 64.GB"],[5,"On an HPC cluster","on-an-hpc-cluster","Each task becomes its own scheduler job, up to 200 at once (the checks, the versions and the parameters run on the launching node), so Bakta, miniprot and the correction run side by side across your samples. eggNOG is one job for all of them. Defaults reach 48 CPUs and 128 GB per job, and every process has a time limit, so a stuck job ends instead of holding its slot. To match your site's queues and accounting: HPC clusters goes into the details that actually bite: shared filesystems, scratch, and where the eggNOG database should live. Large datasets Past about twenty genomes, use a scheduler. The eggNOG step in particular runs once over all the samples rather than once per sample, precisely so a shared filesystem is not hammered by hundreds of jobs reading the same Diamond database.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv --max_cpus 64 --max_memory 256.GB -process.queue long -process.clusterOptions '--account=mygroup'"],[5,"Non-MTBC organisms","non-mtbc-organisms","Lineage typing and Mycobrowser categories are specific to the Mycobacterium tuberculosis complex (MTBC). Turn them off and bring your own reference: See custom reference.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --skip_typing --skip_mycobrowser --genus Escherichia --species \"Escherichia coli\" --ref_annotation my_reference.gbk --ref_genome my_reference.fasta"],[5,"Resuming","resuming","Nextflow caches every completed task, so an interrupted run picks up where it stopped: Keep the parameters identical. Changing an input invalidates that task and everything downstream, which is correct behaviour but does mean the work is redone. The cache lives in the work directory, so do not delete it between a run and its resume.",1,"nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv -resume"],[5,"Running only part of the pipeline","running-only-part-of-the-pipeline","--skip_functional_annotation stops after the correction step. You get the corrected genes as a table with their proteins and sequences (*.AMAP.tsv, .faa, .fna) and the case tables, and skip eggNOG, the duplications and the pangenome: no final_report.html; amap_report.html is still written, without its eggNOG sections. The GFF3 and GenBank files are written after eggNOG, so they are skipped too. Much faster when the corrected genes are all you are after:",1,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --skip_functional_annotation"],[5,"Pinning the pipeline","pinning-the-pipeline","These pages describe AMAP as it is on main, which moves. Without -r, Nextflow runs the copy of AMAP it downloaded first and keeps it until you update it, so two machines can run different code under the same command. For an analysis you will publish, pin both the code and the image: -r takes a branch, a tag or a commit. Give a commit as its full hash: Nextflow 26.10 refuses a short one, with \"Remote does not have 947a99d available for fetch\", where 24.04 takes it. The image is pinned by its digest, which never moves, where a tag such as latest can; this one is the image behind the runs on these pages, and docker image inspect paururo/amap:latest --format '{{index .RepoDigests 0}}' prints the one you have. To move to the newest code, nextflow pull PathoGenOmics-Lab/AMAP. To see which code a run used, open pipeline_info/execution_report_*.html: it names the revision and its commit hash, as in revision main (commit hash 1a1b3959...).",1,"nextflow run PathoGenOmics-Lab/AMAP -r 947a99dee49d2a2429e014296ac35ba08e911108 -profile docker --tsv samples.tsv -process.container 'paururo/amap@sha256:08bd17311ab323317ceecba6e5bc639a29f4c54c9d95f9a7c48ba80a9fc640ee'"],[5,"What a run produces","what-a-run-produces","Beyond the results themselves, results/pipeline_info/ holds the execution report, the timeline, the DAG, a trace with per-process resource usage, software_versions.yml with the exact version of every tool, and params.yml with the parameters used. Those last two are what you want when writing up methods.",1],[5,"Next steps","next-steps","Outputs: what every file is Troubleshooting: when a run fails",1],[5,"Large datasets","note-large-datasets","Past about twenty genomes, use a scheduler.",1],[6,"","","Everything lands under --outdir, results/ by default, in one subdirectory per pipeline stage. {lineage}_{sampleId} below is the sample's name as lineage typing leaves it, L4_G26689 for sample G26689, or the sample ID alone under --skip_typing.",0],[6,"Where to look first","where-to-look-first","The whole analysis, after a run with eggNOG (the full report): results/final_report.html. Each genome and each gene, question by question, in every run (the report by question): results/amap_report.html. Corrected annotation for a sample, after a run with eggNOG: results/AMAP/{lineage}_{sampleId}/*-AMAP.gff3. Proteins for downstream tools, after a run with eggNOG: results/AMAP/{lineage}_{sampleId}/*-AMAP.clean.faa. Everything known about every gene, after a run with eggNOG: results/AMAP/{lineage}_{sampleId}/*-AMAPmastertable.tsv. The corrected genes, their proteins and sequences, in every run: results/AMAP/{lineage}_{sampleId}/*.AMAP.tsv, .faa, .fna. Tool versions for your methods section: results/pipeline_info/software_versions.yml.",1],[6,"The full tree","the-full-tree","The .AMAP. files, with a dot, are the correction's own output, written in every run, --skip_functional_annotation included. The -AMAP. files, with a hyphen, are the finished annotation, written after eggNOG. *.AMAP.faa and *-AMAP.faa hold the same proteins, and the two .fna files the same sequences.",1,"results/ AMAP/ {lineage}_{sampleId}/ *.AMAP.tsv Every run: the corrected genes, one row each *.AMAP.faa their proteins *.AMAP.fna nucleotide sequences *-AMAP.faa Corrected proteins, pseudogenes included *-AMAP.clean.faa No pseudogenes, no internal stops *-AMAP.fna *-AMAP.tsv The finished annotation, every feature *-AMAP.gff3 GFF3 *-AMAP.gbk GenBank *-AMAP.qc.tsv Size, GC%, coding density, annotation rate *-AMAPmastertable.tsv gene, with eggNOG and Mycobrowser *-AMAP.COG.tsv COG category counts *-AMAP.MycoBrowser.tsv *-AMAP.noRef.tsv genes that name reference gene BAKTA/ Bakta's own uncorrected *.gff3 *.gbff *.embl *.tsv *.json *.faa *.ffn *.fna *.hypotheticals.faa *.hypotheticals.tsv *.inference.tsv *.png *.svg Genome map *.txt *.log Bakta summary statistics log CDS_CASES/ summary/ *-summary.tsv Case counts, all samples *-case[1-4].tsv One table per case *-rvmissing.tsv Reference missed, put back *.html Interactive overlap plots *_corrected.tsv As classified, before stop codon check DUPLICATIONS/ duplications_* Diamond hits within genome, sorted duplications_*_groups.tsv Duplicated grouped into families duplications_*_network.png drawn *_interactive.html same, interactive homologous_communities.tsv Communities across samples, metrics homologous_communities.html network PANGENE/ pangenome.gfa Pangenome graph of whole run pangenome.bubbles.txt Structural variation in pangenome.Rtab Gene presence/absence, a column sample pangenome.stat Graph PATHOTYPR/ typing/*_summary.tsv Lineage calls, contig markers fastas/*.fasta Each genome as {lineage}_{sampleId}.fasta pipeline_info/ execution_report_*.html Nextflow execution report execution_timeline_*.html Timeline pipeline_dag_*.html Dependency execution_trace_*.txt Per-process resource usage software_versions.yml .tsv Exact tool versions params.yml Parameters used for this validation.log checks' verdict, when they pass run_summary.txt Status, duration, task command final_report.html full report, after amap_report.html by question,"],[6,"The key files","the-key-files","",1],[6,"*-AMAP.gff3 and *-AMAP.gbk","-amapgff3-and--amapgbk","The corrected annotation, which is the point of the pipeline. The GFF3 is spec-compliant: gene features parent their CDS features, attributes are URL-escaped, ##sequence-region pragmas are emitted per contig, and eggNOG annotation rides along as Ontology_term, eC_number, COG_category, KEGG_ko, KEGG_Pathway and PFAMs. The sequences are appended after ##FASTA. Two kinds of gene carry marks, written the way NCBI writes them: Pseudogenes have pseudo=true in the GFF3 and /pseudo in the GenBank, with the reason in Note and /note: internal stop codons, a frameshift or lost stop that makes the frame run on past the end of the reference protein, no stop at all, or a stop the reference itself lacks. A /pseudo CDS has no /translation. Genes a contig end cut short, in a draft assembly, have partial=true, with start_range=.,START or end_range=END,. on the open end, and < or > on that end of the GenBank location. They are not pseudogenes. Each piece of a cut gene is there under the gene's name, the first in the assembly with the gene's identifier and the others with _2, _3. The same three facts, pseudogene, partial (5', 3' or both) and note, are the last columns of the master table and of *-AMAP.tsv. A gene across the origin of a circular sequence is written the way Bakta writes it. Bakta takes a sequence for a circular replicon when its FASTA description says complete, as NCBI's \"complete genome\" and \"complete sequence\" do, or circular=true, as Unicycler's does. In -AMAP.tsv and the master table such a gene has its start after its end (stcE on plasmid pO157 of *E. coli O157:H7 Sakai is 92,563 to 2,502); in the GenBank it is join(92563..92721,1..2502) in a record marked circular; and in the GFF3 it ends past the sequence's last base, by the sequence's length (92,563 to 95,223 on the 92,721 bp plasmid), on a region feature marked Is_circular=true, as the GFF3 specification and Bakta's own GFF3 have it.",1],[6,"*-AMAP.faa versus *-AMAP.clean.faa","-amapfaa-versus--amapcleanfaa","*-AMAP.faa mirrors the annotation exactly, pseudogenes included, so gene counts match the GFF3. *-AMAP.clean.faa drops the genes AMAP calls pseudogenes, those with pseudo=true in the GFF3, and any sequence with an internal stop codon, and strips the terminal stop. This is the file for orthology, phylogenomics, pangenome tools and dN/dS, all of which expect valid translations. It has fewer sequences than the GFF3 has genes, by design.",1],[6,"*-AMAPmastertable.tsv","-amapmastertabletsv","One row per gene, joining the corrected coordinates, the Bakta annotation, the eggNOG assignment (COG category, description, preferred name, GO, EC, KEGG, PFAMs, CAZy) and, for the Mycobacterium tuberculosis complex (MTBC), the Mycobrowser functional category. This is the table to load when you want to ask a question about the annotation.",1],[6,"*-AMAP.qc.tsv","-amapqctsv","One row per sample: genome size, contig count, N50, GC%, total CDS, how many of them AMAP calls pseudogenes (those with pseudo=true in the GFF3), hypothetical protein percentage, mean and median gene length, coding density, and the eggNOG annotation rate. Useful for spotting the sample that went wrong.",1],[6,"CDS_CASES/","cds_cases","The audit trail for the correction. Each case table lists the genes that fell into it, so you can see exactly what was changed and why. The correction algorithm explains the cases; *-summary.tsv aggregates the counts across samples and feeds the report. Its last three columns say which sample the counts are for: Sequence is the name its files carry (L4_G26689), Lineage what typing called it (Untyped under --skip_typing) and Sample its ID in the sample sheet (G26689).",1],[6,"What a healthy summary looks like","what-a-healthy-summary-looks-like","Two real runs, those of the tutorial, to calibrate expectations. On the left, M. tuberculosis CDC1551 against the bundled H37Rv reference. On the right, E. coli O157:H7 Sakai against K-12 MG1655, using a custom reference. case2, perfect match: 2,813, 3,324. case3, imperfect match: 24, 31. case4, conflicting overlap, resolved: 1,100, 507. case1_kept, genuinely novel: 11, 1,387. rv_missing, rescued: 97, 133. total_cds before correction: 4,077, 5,367. The one number that differs by two orders of magnitude is case 1, and that is biology: CDC1551 is the same species as the reference so almost nothing is genuinely new, while O157:H7 carries prophages and the LEE island that K-12 does not have. What should worry you is the opposite: case 2 at or near zero. Every gene disagreeing with the reference by a few bases means something systematic is wrong with the reference, not with 4,000 individual genes. The usual cause is a reference proteome whose coordinates do not describe the sequence its proteins were translated from.",1],[6,"PANGENE/pangenome.Rtab","pangenepangenomertab","The gene presence/absence matrix of the whole run, the one both reports draw the pangenome from: one row per gene, one column per sample, 1 where pangene found the gene in that sample's genome. Every genome is searched with the same proteins, the reference proteome and the genes no reference gene matches, so a gene with a 1 in every column is one all the samples share. A gene is named after its reference gene (Rv1506c) or, when no reference gene matches it, after one sample's copy of it (L4_CDC1551_01396, the ID in that sample's master table).",1],[6,"final_report.html","final_reporthtml","Self-contained, so it travels by email. Covers per-sample quality, missing reference genes by lineage, PE/PPE family distribution, KEGG pathways, duplication communities and network topology, the pangenome (core, softcore, shell, cloud, frequency spectrum, presence/absence heatmaps), COG and Mycobrowser distributions, case statistics and the associated tests, plus a searchable master-table explorer.",1],[6,"amap_report.html","amap_reporthtml","One self-contained file, organised by the questions a run raises: how each genome annotated, where genes are missing or broken on the reference, one gene in every genome, what the correction changed, the pseudogenes by lineage, what lies outside the reference, and the functional profile. Written with or without the functional annotation. The report by question goes through it.",1],[6,"Effect of the skip options","effect-of-the-skip-options","--skip_typing: No PATHOTYPR/ directory. Samples keep their original IDs instead of being renamed by lineage. --skip_functional_annotation: The run stops after correction. No DUPLICATIONS/, PANGENE/ or final_report.html, and AMAP/ holds only the corrected table and sequences, *.AMAP.tsv, *.AMAP.faa and *.AMAP.fna, without the GFF3, the GenBank or the master table. You still get BAKTA/ and CDS_CASES/, including *_corrected.tsv with the classification's coordinates, and amap_report.html, without its eggNOG sections. --skip_mycobrowser: *-AMAP.MycoBrowser.tsv is written empty and the Mycobrowser sections drop out of the report. Everything else is unaffected.",1],[6,"Next steps","next-steps","Overview: what produced all this Duplication metrics: reading the network numbers",1],[7,"","","A run with the functional annotation ends in results/final_report.html, the whole analysis in one file, built after eggNOG, the duplications and the pangenome. Every run, with or without them, also writes the report by question, results/amap_report.html. So a run with --skip_functional_annotation has no final_report.html; amap_report.html is still written, without its eggNOG sections. It is self-contained: all the figures, tables and JavaScript are embedded, so it opens with no network and survives being emailed to someone who has never heard of Nextflow. To see what the sections look like before running anything, open the example report: a rendered report for twelve genomes of the Mycobacterium tuberculosis complex (MTBC) across four lineages. The example uses generated data The twelve samples in it are synthetic. The distributions are drawn around a real M. tuberculosis CDC1551 run, so the shapes are representative, but no conclusion should be drawn from the numbers themselves. bin/report/demo/make_demo.py builds that dataset if you want to render the report yourself without waiting for a pipeline run.",0],[7,"What is in it","what-is-in-it","",1],[7,"Overview","overview","Dataset composition, then a per-sample quality panel: CDS counts, correction rates and missing reference genes. The annotation quality tab carries coding density, hypothetical percentage and genome size, which is where an outlier sample declares itself before you have gone any further.",1],[7,"Annotation results","annotation-results","CDS distribution per lineage, the genes with no counterpart in the reference, and the correction summary: how many genes fell into each case and what the adjudication did with them. Read this alongside the correction algorithm. Missing reference genes get both a heatmap, for spotting a gene absent from a whole lineage rather than one sample, and a searchable table.",1],[7,"Functional annotation","functional-annotation","COG categories from eggNOG, Mycobrowser functional categories for MTBC, and a comparison against the reference distribution. Each has its own per-category breakdown in counts and percentages, with the statistical tests alongside. PE/PPE gene families get their own section, since they are both the hardest part of an MTBC genome to annotate and usually the most interesting. The KEGG pathways follow: the most represented metabolic pathways, and how they differ between lineages. The pseudogenes close it: how many AMAP called in each sample, lineage by lineage, a sample with none at 0, the same genes the QC table counts and the GFF3 marks pseudo=true. The master table's note column says why each is a pseudogene.",1],[7,"Duplications","duplications","Community sizes, cross-genome homologous pairs and the network topology metrics. Duplication metrics explains what each number means.",1],[7,"Pangenome","pangenome","Core, softcore, shell and cloud classification, the gene frequency spectrum, and presence/absence heatmaps.",1],[7,"CDS statistics","cds-statistics","How many genes fall in each case, lineage by lineage: box and violin plots with Dunn's tests between lineages, the share of each case in every lineage, a heatmap of the means, and a table per case with the count, mean, median and spread of each lineage.",1],[7,"Master table explorer","master-table-explorer","Every gene of every sample in one searchable, sortable table, with the functional annotation attached. Useful for answering a specific question without leaving the report.",1],[7,"What changes when steps are skipped","what-changes-when-steps-are-skipped","--skip_typing: Everything still renders, but every sample is grouped under Untyped, so the per-lineage comparisons have a single group and the statistical tests drop out. --skip_mycobrowser: The Mycobrowser and PE/PPE sections are omitted. COG, KEGG, pangenome and duplications are unaffected. This is automatic for non-MTBC organisms. --skip_functional_annotation: The run stops after correction: no final_report.html; amap_report.html is still written, without its eggNOG sections.",1],[7,"Rendering it yourself","rendering-it-yourself","The report is bin/report/report_AMAP.Rmd, rendered by the FINAL_REPORT process. To iterate on it without running the pipeline: Keep it self-contained Nothing in the Rmd, header.html or styles.css may point at a remote URL. Pandoc fetches and inlines a remote resource when the machine rendering has a network, which hides the problem, and leaves it dangling when it does not. A report rendered on an isolated compute node then arrives with a broken image and a browser trying to reach the internet. Both logos live next to the Rmd for this reason, and the image zoom is plain DOM rather than jQuery from a CDN.",1,"python3 bin/report/demo/make_demo.py demo_inputs docker run --rm -v \"$PWD:/w\" -w /w paururo/amap:latest Rscript -e 'rmarkdown::render(\"bin/report/report_AMAP.Rmd\", output_file=\"/w/demo.html\", params=list( version=\"v1.0.1\", cog_file=\"/w/demo_inputs/input_cog.txt\", myco_file=\"/w/demo_inputs/input_myco.txt\", summary_file=\"/w/demo_inputs/input_summary.txt\", empty_file=\"/w/demo_inputs/input_noref.txt\", rvmissing_file=\"/w/demo_inputs/input_rvmissing.txt\", mastertable_file=\"/w/demo_inputs/input_mastertables.txt\", pangene_file=\"/w/demo_inputs/input_pangene.txt\", dup_file=\"/w/demo_inputs/homologous_communities.tsv\", qc_file=\"/w/demo_inputs/input_qc.txt\"))'"],[7,"The example uses generated data","note-the-example-uses-generated-data","The twelve samples in it are synthetic.",1],[7,"Keep it self-contained","note-keep-it-self-contained","Nothing in the Rmd, header.html or styles.css may point at a remote URL.",1],[8,"","","Every run writes results/amap_report.html, next to the full report, final_report.html, when the run has one: one file that answers, section by section, the questions a run raises. Did each genome annotate well? Where on the reference are genes missing or broken? What happened to one gene in every genome? What did the correction change? Every table filters, sorts and saves as CSV, every figure saves as SVG, and a click on a genome or a gene opens it. It is written with or without the functional annotation. It needs only what the correction leaves, so a run with --skip_functional_annotation has no final_report.html; amap_report.html is still written, without its eggNOG sections, which say that the run skipped them. It is self-contained: the data travels inside the file, compressed, and nothing is fetched from anywhere, so it opens with no network and survives being emailed. Fifteen genomes of the Mycobacterium tuberculosis complex (MTBC) make a file of about half a megabyte, against 4.5 MB for final_report.html with twelve. It is dark, like the website, with a switch to light at the top right.",0],[8,"The sections","the-sections","",1],[8,"1. Did each genome annotate well?","1-did-each-genome-annotate-well","One row per genome: lineage, genome size, contigs and N50, Bakta's calls and AMAP's genes, the share of the reference genes it has, and how many are missing, pseudogenes, partial, found more than once or outside the reference. A value far from those of comparable genomes is flagged (see How a value is flagged). A click on a genome opens it: its numbers, and its genes in each state, each list searchable and saved as CSV.",1],[8,"2. Where on the reference are genes missing or broken?","2-where-on-the-reference-are-genes-missing-or-broken","Every reference gene at its place on the reference, one row per genome, the genomes grouped by lineage. A missing gene fills its row; a pseudogene is a band through the middle; a second copy a band on top. Deletions that genomes share show as blocks: with MTBC genomes, the regions of difference. Drag across the map to zoom into a region; its genes are then listed with how many genomes have each in each state. Below the map, Blocks of missing genes lists every run of two or more consecutive reference genes missing from a genome, grouped where genomes overlap, and says which lineages lack each block in all their genomes.",1],[8,"3. What happened to one gene in every genome?","3-what-happened-to-one-gene-in-every-genome","Type a locus, a gene name or part of a product. The card gives the reference's own coordinates, length, product, functional category and, from Mycobrowser, whether the reference calls it a pseudogene and which drugs it has resistance mutations for. The table gives, for every genome, its state, locus tag, contig, coordinates, strand, length against the reference, the note of a pseudogene and how AMAP got the gene.",1],[8,"4. What did AMAP change in Bakta's annotation?","4-what-did-amap-change-in-baktas-annotation","A waterfall from Bakta's calls to AMAP's genes: the case 1 calls the filter removed, the case 4 calls the reference replaced, then the genes placed from the reference over a call and those added where Bakta called nothing. For one genome a click on a bar lists its genes; for the calls AMAP did not keep, Bakta's coordinates next to those of the gene that took their place. The correction algorithm says what each case means.",1],[8,"5. Which genes are broken, and in which lineages?","5-which-genes-are-broken-and-in-which-lineages","Every gene AMAP flags as a pseudogene in at least one genome, against the genomes, with the reason. First the genes broken in every genome of some lineages and in no other, then those broken in every genome, then those broken in all but one or two, then the rest. With Mycobrowser, each is compared with the reference's own pseudogenes.",1],[8,"6. What do the genomes carry that the reference does not?","6-what-do-the-genomes-carry-that-the-reference-does-not","How many genomes carry each reference gene, with the genes some genomes lack; the genes that name no reference gene, grouped by product across genomes; the reference genes with more than one copy; and, when the run has them, the pangenome and the homologous communities.",1],[8,"7. Do the lineages differ in function?","7-do-the-lineages-differ-in-function","The genes each lineage loses (missing or pseudogene) by Mycobrowser category, and with the functional annotation, genes by COG category and KEGG pathway. One line per category, a dot per lineage, the categories that differ most between lineages first.",1],[8,"8. How was this made?","8-how-was-this-made","AMAP's version, the command, the reference, every parameter (pipeline_info/params.yml), every tool's version (pipeline_info/software_versions.tsv), and what each state means. The bar under the top one filters every section at once: a lineage's pill shows or hides it (Alt or Cmd and a click keeps that lineage alone), and the box keeps the genomes whose name contains what is typed.",1],[8,"Gene states","gene-states","Each reference gene gets one state in each genome, from the genes that name it in the genome's final annotation, AMAP/{lineage}{sampleId}/{lineage}{sampleId}.AMAP.tsv. Intact: A gene names it, and nothing below applies. Missing: No gene names it. Pseudogene: The best gene naming it is flagged as a pseudogene: an internal stop, a lost stop or a frame that runs on past the reference protein. Partial: The best gene naming it runs into a contig end and is cut there. More copies: More than one gene names it alone. Length differs: One intact copy, of another length than the reference gene. Shared copy: Only a gene that names several reference genes at once names it, such as an IS6110 copy that matches all seventeen; counted as found. When several apply, the first in the table wins: a pseudogene that is also cut by a contig end is a pseudogene. \"Reference genes found\" counts every state but missing.",1],[8,"How a value is flagged","how-a-value-is-flagged","Each genome is compared with its own lineage when that lineage has five genomes or more, and with every other genome otherwise. A value is flagged when it lies more than 3.5 robust standard deviations (1.4826 times the median absolute deviation) from their median, on the side that is bad: more missing genes, pseudogenes, partial genes, contigs or genes outside the reference, fewer reference genes found, a smaller N50, a genome size either way. Each also has to be further from the median than a minimum (five genes, five contigs, half a point of reference genes found, 1% of the genome size or 5% of the N50), so that identical genomes do not flag a gene or two of difference. Fewer than four genomes are not compared at all. A flag says a genome is unusual, not that it is wrong. M. bovis among M. tuberculosis genomes is flagged for its missing genes, which are its regions of difference. A genome with twice the pseudogenes of the others is worth a look at its assembly: frameshifts from sequencing errors look like pseudogenes.",1],[8,"What each section needs","what-each-section-needs","*.AMAP.tsv: FIX_CDS, No report. The reference proteome: the reference, No report. Case summaries and tables: TYPE_CASES, No lineages, no correction section. Bakta's folder: BAKTA_ANNOTATOR, No genome sizes, contigs or products of Bakta's calls. Mycobrowser table: the reference, MTBC only, Names and products from the genomes' own annotation; no categories or drug resistance. Master tables: GET_OUTPUTS, functional annotation, No COG categories or KEGG pathways. Pangenome matrix: REPORT_INPUTS, functional annotation, No pangenome. Homologous communities: GET_ALLDUP, functional annotation, No communities. params.yml, software_versions.tsv: DUMP_PARAMS, SOFTWARE_VERSIONS, No parameters or versions. A section without its inputs says which and why instead of drawing anything. With a custom reference, under --skip_typing and --skip_mycobrowser, every genome is Untyped and the names and products come from Bakta and eggNOG. A genome that was classified but has no corrected genes, because the pipeline set aside its correction after it kept failing, is named at the top of the report. One whose Bakta, miniprot or classification task was set aside never reaches the report: see a sample is missing.",1],[8,"Size","size","The data of each genome is a few kilobytes: its genes are stored as how far each lies from where the previous gene and the reference predict it, which for a genome collinear with its reference is mostly nothing. Up to 100 genomes, the report also lists every Bakta call AMAP did not keep, with its coordinates; past that it only counts them, to keep the file small, and CDS_CASES/{lineage}_{sampleId}/ has them for any run. 15: 0.5 MB, measured. 100: 1.5 to 1.7 MB. 300: 1.9 to 2.1 MB. 500: 2.9 to 3.3 MB. The sizes past fifteen are worked out from the fifteen MTBC genomes of eight lineages the report was measured on, the larger figure with each genome's data compressed on its own and the smaller with the sharing measured between those fifteen. In the browser, five hundred genomes open in about three seconds.",1],[8,"Writing one by hand","writing-one-by-hand","bin/report/amap_report.py needs only Python 3.9 or later. Pointed at a results/ folder it reads the files where the pipeline publishes them: The reference proteome is the one the run used: for H37Rv, the AMAP container has it at /reference/aa_H37Rv.translated.fas, and the Mycobrowser table at /reference/Mycobacterium_tuberculosis_H37Rv_txt_v4.txt. The page reads its data with the browser's own decompression, which every current browser has: Chrome and Edge 80, Firefox 113 and Safari 16.4, and later. An older one says so instead of drawing the report.",1,"python3 bin/report/amap_report.py --amap results/AMAP --cases results/CDS_CASES --bakta results/BAKTA --master --ref-proteome ref_proteome.faa --mycobrowser Mycobacterium_tuberculosis_H37Rv_txt_v4.txt --versions results/pipeline_info/software_versions.tsv --params results/pipeline_info/params.yml --out amap_report.html"],[9,"","","AMAP corrects an annotation by refereeing it against a trusted one. For the Mycobacterium tuberculosis complex (MTBC) that trusted annotation is H37Rv, bundled in the image. Point --ref_annotation and --ref_genome at your own and the same machinery works for any prokaryote. Both must be given together. Setting one without the other stops the run immediately.",0,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --skip_typing --skip_mycobrowser --genus Escherichia --species \"Escherichia coli\" --ref_annotation my_reference.gbk --ref_genome my_reference.fasta"],[9,"What counts as a good reference","what-counts-as-a-good-reference","A closed, well-curated genome of a close relative, ideally the type strain or a reference assembly from RefSeq. The correction assumes the reference genes are right, so its quality is the ceiling on your result. --ref_annotation: GFF3 (.gff, .gff3) or GenBank (.gbk, .gb, .genbank, .gbff). Detected from the extension, with GenBank tried first for anything else. --ref_genome: FASTA of the same genome. It must be the same assembly the annotation describes, since coordinates are taken at face value.",1],[9,"What AMAP does with it","what-amap-does-with-it","PREPARE_REFERENCE runs once, before any sample is processed, and does three things. 1. Extracts the reference proteome. Every CDS becomes a protein, headed GENE_ID_START-END contig=SEQUENCE, SEQUENCE being the name --ref_genome gives the sequence the gene is on. The header matters. The paralog step reads its coordinates back to predict where each protein belongs in the sample, which tells its own locus from a paralog's. From GenBank, the existing /translation qualifier is used when present; otherwise the sequence is extracted and translated. From GFF3, the lines of each CDS are joined first, as GenBank's join() is, and translated with the bacterial code, reading transl_except (a selenocysteine) where it is given. A GFF3 protein under 8 amino acids is left out. The gene ID comes from locus_tag, then gene, then protein_id for GenBank; locus_tag, ID, gene, then Name for GFF3. A CDS with none of these is skipped. Genes across the origin A CDS across the origin of a circular sequence ends past the sequence's last base, the way GFF3 writes it: tagA on plasmid pO157 (92,721 bp) of E. coli O157:H7 Sakai, join(92527..92721,1..2502) in its GenBank record, is pO157p01_92527-95223. On E. coli K-12 the two formats give the same 4,318 proteins, and on Sakai the same 5,155. 2. Annotates the reference genome with Bakta. The same Bakta that will annotate your samples, run on a genome whose true annotation is known, keeping the FASTA's names for its sequences (--keep-contig-headers). 3. Works out what Bakta invents. Any Bakta CDS that does not overlap a curated gene of its own sequence is a false positive on this genome, and by extension a call Bakta is prone to make on your samples too. These become the bad_bakta set used to filter novel-gene candidates in case 1. Bakta writes a call across the origin of a sequence whose FASTA description says complete from its start to its stop, start after end (stcE on pO157 is 92563..2502), and it is compared with the genes on both sides of the origin. The calls on a sequence the annotation has no gene on are not judged: the annotation says nothing about them. For MTBC this set was curated by hand. With a custom reference AMAP derives the equivalent automatically, which is what makes organism-agnostic mode work rather than merely run.",1],[9,"What you must turn off","what-you-must-turn-off","--skip_typing: pathotypr assigns MTBC lineages from tuberculosis-specific marker SNPs. Meaningless elsewhere, and it will fail. AMAP warns when --genus is not Mycobacterium and this is not set. --skip_mycobrowser: Mycobrowser functional categories are an MTBC resource. Auto-enabled when --ref_annotation is set and --genus is not Mycobacterium, but setting it explicitly is clearer. Also set --genus and --species to match your organism, since Bakta uses them for naming and reference selection.",1],[9,"What stays the same","what-stays-the-same","Everything else. Case classification, conflict resolution, missing-gene rescue, the stop codon check, eggNOG functional annotation, duplication analysis, the pangenome and the report all work identically. The report simply omits the lineage and Mycobrowser sections, and the pangenome calls each genome by its sample ID where a typed run would use its lineage name.",1],[9,"What to expect from a real run","what-to-expect-from-a-real-run","E. coli O157:H7 Sakai corrected against K-12 MG1655, both straight from RefSeq, in the tutorial's run: The 142 are the automatically derived equivalent of the hand-curated MTBC false-positive list, and 1,387 case 1 genes is what you would expect between two E. coli strains that genuinely differ. If your case 2 comes out at or near zero, stop and look at the reference rather than the sample. Every gene missing an exact match by a constant offset points at the reference, not at 4,000 coincidences.",1,"reference proteins extracted from the K-12 GenBank 4318 CDS Bakta called on that same genome 4296 false positives derived difference 142 case2 3324 case3 31 case4 507 case1_kept 1387 rv_missing 133 final annotation 5335 genes"],[9,"Worked example","worked-example","E. coli K-12 MG1655 against the RefSeq reference:",1,"Fetch the reference datasets download genome accession GCF_000005845.2 --include genome,gff3 unzip ncbi_dataset.zip nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv --skip_typing --skip_mycobrowser --genus Escherichia --species \"Escherichia coli\" --ref_annotation ncbi_dataset/data/GCF_000005845.2/genomic.gff --ref_genome ncbi_dataset/data/GCF_000005845.2/GCF_000005845.2_ASM584v2_genomic.fna"],[9,"Things that trip people up","things-that-trip-people-up","The annotation and the genome must match Coordinates from the annotation are used directly against the FASTA. An annotation from a different assembly of the same organism produces nonsense: extracted proteins will not translate, and the reference proteome will be mostly empty. PREPARE_REFERENCE writes the count of extracted proteins into its work directory, where grep -h 'Curated proteins' work///.command.out finds it, so check it looks like a genome's worth. Reference genes need identifiers A GFF3 whose CDS features carry no locus_tag, ID, gene or Name is skipped entirely, and the run fails with \"No proteins extracted\". This happens with minimal GFF3 exports from some annotation tools. Distance to the reference The correction expects your samples to be the same species as the reference. The further away they are, the more genes fall into case 1 (nothing in the reference to compare against) and the less AMAP has to say. It is a correction tool, not an ortholog finder.",1],[9,"Next steps","next-steps","Overview: what the correction does with the reference Parameters: the full flag reference",1],[9,"Genes across the origin","note-genes-across-the-origin","A CDS across the origin of a circular sequence ends past the sequence's last base, the way GFF3 writes it:",1],[9,"The annotation and the genome must match","note-the-annotation-and-the-genome-must-match","Coordinates from the annotation are used directly against the FASTA.",1],[9,"Reference genes need identifiers","note-reference-genes-need-identifiers","A GFF3 whose CDS features carry no locus_tag, ID, gene or Name is skipped entirely, and the run fails with \"No proteins extracted\".",1],[9,"Distance to the reference","note-distance-to-the-reference","The correction expects your samples to be the same species as the reference.",1],[10,"","","AMAP does not annotate genomes from scratch. It takes an annotation that already exists, compares it against one that is trusted, and adjudicates the differences. The premise is that de novo annotators are good at finding genes and less good at their exact boundaries. Bakta will call a start codon a few codons late, or split one gene into two, or invent a short ORF that is not really there. If a curated reference for a close relative exists, those errors are visible: align the reference proteins onto the new genome, and where the two disagree, decide which is more likely right. On genomes of the Mycobacterium tuberculosis complex (MTBC) that adjudication changes roughly 30% of genes.",0],[10,"The three inputs to a decision","the-three-inputs-to-a-decision","The Bakta annotation. Every CDS Bakta called on your genome, with coordinates and a locus tag. The miniprot alignments. Every reference protein aligned onto your genome, with the coordinates where it landed. The reference proteins carry their own coordinates in their headers (Rv0001_1-1524), which lets a later step tell a real locus from a paralogous placement. The false-positive set. Proteins that a de novo annotator calls on the reference genome itself but that are not in the curated annotation. Bakta's characteristic mistakes, in other words. For MTBC this is a curated file; with a custom reference AMAP derives it by annotating the reference and taking the difference.",1],[10,"The pipeline","the-pipeline","Eleven steps, from genomes to reports: eggNOG database setup, skipped if it is already built Lineage typing with pathotypr, MTBC only Annotation with Bakta Protein alignment with miniprot Case classification: every CDS sorted by how it relates to the reference Correction: conflicts resolved, missing genes rescued, stop codons checked Functional annotation with eggNOG-mapper Duplication analysis: within and across genomes Pangenome with pangene The full report, final_report.html, after eggNOG The report by question, amap_report.html, in every run The workflow page walks through each with its inputs and outputs.",1],[10,"The correction itself","the-correction-itself","Steps 5 and 6 are what makes AMAP more than a wrapper. Every CDS is sorted into one of five outcomes: Case 1: Bakta called it, the reference has nothing there. Potentially novel. Case 2: Coordinates match the reference exactly. Kept. Case 3: Coordinates match but alignment length differs. Kept. Case 4: Overlaps a reference gene, but the coordinates conflict. Must be resolved. Missing: The reference has a gene there, Bakta missed it. Rescued. Cases 1 and 4 are where the work is: deciding which novel genes are real, and which set of coordinates wins a conflict. The correction algorithm gives the rules and the thresholds.",1],[10,"Next steps","next-steps","The workflow: the eleven steps in detail The correction algorithm: case classification, conflict resolution, stop codon check Duplication metrics: what the network numbers mean",1],[11,"","","The pipeline as a metro map: stations are the processes of main.nf, lines are what travels between them. Pick a station for what it does and where its files go, or switch an option to see what a run leaves out.",0],[11,"1. eggNOG database setup","1-eggnog-database-setup","Checks whether the eggNOG database exists at --eggnog_db and builds it if not. A bacteria.dmnd under 100 MB is treated as a truncated download and rebuilt. The database is large, so point --eggnog_db at persistent shared storage on a cluster. Skipped entirely under --skip_functional_annotation.",1],[11,"2. Lineage typing","2-lineage-typing","pathotypr assigns each genome to a lineage of the Mycobacterium tuberculosis complex (MTBC) using phylogenetic marker SNPs, and the genomes are renamed after their sample ID with the lineage in front (L4_G26689 for sample G26689). That prefix carries through every downstream file and is what lets the report group results by lineage. pathotypr types each contig of a draft on its own, and the lineage of the draft is the one the markers of all its contigs add up to. A genome of one sequence has that sequence renamed after the sample and keeps the description that follows the name in its header, so Bakta still reads a closed genome as complete and circular when the description says so (NCBI's \"complete genome\" and \"complete sequence\", Unicycler's circular=true); the contigs of a draft keep their names. A genome with no lineage markers at all is named NA_G26689. MTBC-specific. Turn it off with --skip_typing for anything else, and sample IDs stay as you wrote them.",1],[11,"3. Annotation with Bakta","3-annotation-with-bakta","Bakta produces the baseline annotation: CDS, rRNA and other features (AMAP runs it with --skip-trna, so there are no tRNAs). This is the annotation AMAP will correct, and it is published unmodified under BAKTA/ so you can compare. --genus and --species are passed through, so setting them correctly improves the starting point.",1],[11,"4. Protein alignment with miniprot","4-protein-alignment-with-miniprot","miniprot aligns the reference proteome onto each genome, spliced and frameshift-aware. Two alignments are produced: the reference proteome, and the false-positive set that will filter novel-gene candidates. Output is PAF. Coordinates are converted from 0-based to 1-based on read, and for a protein hitting several positions, the best hit per position is kept so multi-copy families such as PE/PPE keep all their placements. In a draft assembly the genes a contig end cuts get a second pass. miniprot places the larger piece of such a gene only, so what that placement lacks is aligned again as a protein of its own, and its placement at the end of another contig is the rest of the gene (pieces of cut genes).",1],[11,"5. Case classification","5-case-classification","Every Bakta CDS is compared against the alignments and sorted into cases 1 to 4, plus the reference genes Bakta missed. Comparisons are strand-aware throughout: a gene at (100, 200, +) and one at (100, 200, -) are different genes and are never merged. Tables land in CDS_CASES/, one per case, plus interactive overlap plots. The correction algorithm covers the classification.",1],[11,"6. Correction","6-correction","The cases are resolved into one corrected coordinate set: conflicts adjudicated, novel genes filtered, missing reference genes rescued, duplicates removed. Then check_cds.py translates every surviving CDS. A gene that does not end in a stop codon has its coordinates extended to the next in-frame stop, up to 300 bp. Genes with internal stops are flagged as pseudogenes rather than discarded. Anything that cannot be translated at all is dropped, so the annotation table and the FASTA files always agree. Outputs are the corrected gene table and its protein and nucleotide files under AMAP/ (*.AMAP.tsv, .faa, .fna). The GFF3 and GenBank files follow once eggNOG has run, in step 7.",1],[11,"7. Functional annotation","7-functional-annotation","eggNOG-mapper assigns COG categories, KEGG orthologs and pathways, GO terms, EC numbers, PFAM domains and CAZy families. This runs once over all samples together, not once per sample. With closely related genomes most proteins are identical across samples, so they are deduplicated first, annotated once, and the results propagated back. For a few hundred MTBC genomes that cuts the Diamond search by an order of magnitude and avoids hundreds of jobs contending for the same database on a shared filesystem. The results are merged into *-AMAPmastertable.tsv and into the GFF3 attributes.",1],[11,"8. Duplication analysis","8-duplication-analysis","Each proteome is searched against itself with Diamond to find within-genome duplications, which are grouped into families by connected components. Then the proteins of every sample's families are pooled and searched all against all to build homologous communities across genomes, with network metrics computed per community. Duplication metrics explains the numbers.",1],[11,"9. Pangenome","9-pangenome","pangene builds one pangenome graph for the whole run and a gene presence/absence matrix, which the report turns into core, softcore, shell and cloud classifications, a frequency spectrum and presence/absence heatmaps. Every genome is searched with the same proteins: the reference proteome, and the genes of each sample that no reference gene matches. miniprot aligns them to every sample's genome, its own included: the ones PATHOTYPR renamed or, with --skip_typing, the genomes of the sample sheet, each under its sample ID. A row of the matrix is then a gene, named after its reference gene or, outside the reference, after one sample's copy of it, and a column a sample, so the genes the samples share are the rows found in every column.",1],[11,"10. The full report","10-the-full-report","final_report.html, a single self-contained HTML file built from every output above, so only a run with the functional annotation has it. Sample quality, missing reference genes by lineage, PE/PPE families, KEGG pathways, duplication communities and topology, the pangenome, COG and Mycobrowser distributions, case statistics with the associated tests, and a searchable table of every gene. Alongside it, pipeline_info/ collects the Nextflow execution report, timeline, DAG and trace, plus software_versions.yml and params.yml for your methods section.",1],[11,"11. The report by question","11-the-report-by-question","amap_report.html, one self-contained file organised by question, written in every run: how each genome annotated, where reference genes are missing or broken, one gene in every genome, what the correction changed and the pseudogenes by lineage. It reads the correction's outputs directly, so it is written with or without the functional annotation, and the functional steps add their sections when they run. The report by question goes through it.",1],[11,"Next steps","next-steps","The correction algorithm: the rules behind step 6 Outputs: every file this produces",1],[12,"","","The core of AMAP. It referees between the initial Bakta annotation and the miniprot alignments of a trusted reference proteome, classifying every candidate gene and then applying a set of rules to resolve the disagreements. Coordinates throughout are 1-based inclusive, so a gene's length is end - start + 1. Every comparison is strand-aware and contig-aware: genes on opposite strands at the same position are different genes and never conflict, and neither do genes on different contigs of a draft assembly, where every contig starts at 1. Bakta runs with --keep-contig-headers so that it names the contigs as the FASTA and miniprot do; by default it renames them contig_N. The steps in order: Placements: small proteins found exactly, the pieces of cut genes placed, and a protein's placements on its paralogs' loci dropped. Classification: every Bakta call sorted into a case, and the reference genes Bakta missed. Filtering case 1: a call with nothing in the reference stays only if Bakta is not known to invent it. Resolving case 4: the reference's coordinates win, unless Bakta's call is much longer. Deduplication on insert: a gene displaces one already accepted only when it is much longer. Rescuing missing genes: a reference gene no gene accounts for goes back in. Final deduplication: one place per gene, plus real copies and the pieces of cut genes. Stop codon check: every gene translated, then extended, flagged as a pseudogene or marked partial.",0],[12,"0. Placements","0-placements","",1],[12,"Small proteins","small-proteins","miniprot needs a seed of several residues and enough score to report an alignment, which a protein of a few dozen residues rarely gives it: annotating E. coli K-12 with its own proteome, it placed none of 33 proteins of 8 to 29 residues, and it put the toxin IbsC (b4665), 19 residues three of which differ from its neighbour IbsD's, on that neighbour only. Every protein without an exact, complete placement is looked for in the six-frame translation of the genome, preceded by a start codon and followed by a stop, and each exact copy found where miniprot did not already place the protein becomes a complete placement. Proteins shorter than 8 residues are not searched. A protein of up to 50 residues with no exact copy is looked for again, allowing substitutions in at most a fifth of its residues after the first, which varies with the start codon; a stop codon in place of a residue counts as one, and the gene is then flagged as a pseudogene in step 7. E. coli O157:H7 Sakai carries ten of K-12's micro-proteins with one to four residues changed, and none was annotated. A copy with an insertion or a deletion is not found this way.",1],[12,"Pieces of cut genes","pieces-of-cut-genes","A gene a contig end cuts is placed in pieces. A draft assembly cuts the genes its contig breaks run through, and miniprot reports only the placements within 5% of a protein's best, so it places the larger piece of such a gene and not the smaller one at the end of another contig: in CDC1551 cut into 40 contigs, none of the 25 smaller pieces was annotated. A placement was cut by its contig when all of these hold: it lacks residues at one end of its protein it ends within 30 bp of the contig's edge on that side those residues would not fit in the bases left What it lacks, if 8 residues or more, is aligned again as a protein of its own, and its best placement is the piece at the other contig end. This pass runs with -L 10 -n 2, so that miniprot indexes and chains the short reading frames an edge leaves. A placement of that protein which runs off a contig edge on the side of the cut, within 30 bp, is a piece of the gene and becomes a placement of the gene's protein. In the CDC1551 draft that places 22 of the 25 smaller pieces; the other three are five or six codons long.",1],[12,"Paralog placements","paralog-placements","miniprot places a reference protein everywhere it scores within 5% of its best, so a protein with a paralog is placed at both loci. Before anything is classified, each placement is checked against where the protein belongs. The reference proteome carries each protein's coordinates in its header (Rv1150_1278269-1278820), but those are reference coordinates, and any indel upstream shifts every position after it. So each placement's reference position is predicted from the nearest proteins placed only once, which follow the sample's local offset from the reference, and the placement closest to where the protein belongs is its syntenic one. A locus that several proteins are placed at, exactly, predicts nothing: it is a repeat the reference has in several copies and the sample in one, like the single IS6110 of M. bovis, and the reference positions of its proteins are spread over the whole genome. A placement is then dropped when all of these hold: another protein's syntenic placement covers at least 50% of it, on the same strand and in the same reading frame that protein matches there at least as well the two proteins live in different places in the reference, so they are not simply neighbours that overlap What this removes is a protein sitting on someone else's locus: Rv1945 on Rv1148c, the mce3 genes M. bovis has lost (RD7) on their mce1 paralogs, or the sixteen other IS6110 names on each IS6110 copy. What it leaves alone is a copy at a locus no protein claims, such as an IS6110 insertion at a site the reference lacks or a duplication in the sample, and genes that genuinely overlap, which sit in different frames (Rv1041c and Rv1042c). When two proteins both look syntenic at one locus, which happens when a deletion removes a gene and leaves its paralog next door as the nearest place left, the better match keeps the locus. CDC1551 lacks Rv3427c and Rv3428c, and without this Rv3425 and Rv3426 would sit on Rv3429.",1],[12,"1. Classification","1-classification","Every Bakta CDS is compared against the miniprot alignments. Case 2, perfect match. Same start, same end, same strand as an alignment, and the same length. Correct, kept as is. Case 3, imperfect match. Same coordinates, but the aligned length differs, typically from indels. Reliable enough, kept. Case 4, conflicting overlap. A Bakta CDS and an alignment overlap on the same strand, but their coordinates do not match. This is the conflict that has to be adjudicated. Case 1, potentially novel. A Bakta CDS with no overlap to any alignment. Either a genuine gene absent from the reference, or something Bakta invented. Missing reference genes. An alignment with no overlapping Bakta CDS. The reference has a gene there and the annotator missed it. Alignments with no aligned bases at all are discarded first. Each set is written to its own table under CDS_CASES/, so the whole decision is auditable.",1],[12,"2. Filtering case 1","2-filtering-case-1","Novel-gene candidates are scrutinised before being believed. A candidate is dropped if either holds: it overlaps the false-positive set by 80% or more of its own length, meaning it is a call the annotator is known to make wrongly on the reference genome itself it aligns to the reference at 80% or more match identity, meaning it is not novel at all but a shifted version of a gene that is already there Candidates that pass both, and candidates with no reference alignment to judge by, are kept as case1_kept.",1],[12,"3. Resolving case 4","3-resolving-case-4","",1],[12,"Single overlaps","single-overlaps","One Bakta CDS against one reference alignment, on the same strand of the same contig. Lengths are compared, and: the reference wins by default the Bakta version wins only if it is longer by more than 550 bp, and the alignment is not complete in the call's reading frame The asymmetry is deliberate. A reference alignment is usually the better boundary, but a much longer Bakta call normally means the alignment fragmented, and discarding a full-length gene in favour of a partial alignment would be the worse error. Beyond length: A complete alignment has not fragmented. It covers the whole reference protein down to its stop codon, so a longer call in its frame ends at the same stop and only starts further upstream (plcD, Rv1755c, 843 bp in H37Rv and 1,545 bp in Bakta's call on CDC1551). A call in another frame is another gene, and keeps its chance to win on length; the alignment then comes back in step 5. A call that wins in the alignment's frame is the protein's gene, longer than in the reference, and takes its name in step 5. A Bakta CDS that overlaps more than one alignment loses: the reference wins regardless.",1],[12,"Multiple overlaps","multiple-overlaps","Several Bakta CDS against one alignment, the signature of a gene that has been split. The best-overlapping Bakta CDS is picked first, then compared to the alignment on length: the reference wins if it is at least as long, if the Bakta CDS beats it by less than 500 bp, if the alignment is complete in the call's frame, or if the call overlaps other alignments as well, which makes it several reference genes read as one (Rv3022c and Rv3021c are one reading frame in CDC1551) otherwise the Bakta CDS wins",1],[12,"4. Deduplication on insert","4-deduplication-on-insert","Each winner is checked against the genes already accepted. Where a new coordinate overlaps an existing one by 70% or more on the same strand: the new gene displaces the existing one if it is longer by more than 550 bp it also displaces it if the existing gene is shorter by less than that margin and is not a reference gene otherwise the existing gene stands and the new coordinate is dropped A reference gene is one whose identifier is a reference protein's. The test used to look for coordinates embedded in the identifier, which are stripped as the alignments are read, so no gene qualified and a call of similar length could displace any of them: Bakta's version of Rv1614, one codon longer, replaced it.",1],[12,"5. Rescuing missing genes","5-rescuing-missing-genes","A reference protein that no gene accounts for goes back in, at its best placement. A gene accounts for a placement when it sits on it: on the same contig and strand in the same reading frame, since genes that really overlap sit in different frames (Rv0724A and Rv0725c) covering at least 50% of it and not known to be another reference gene, since some are nested in the same frame (E. coli hokC inside mokC) Placements on other genes' loci have already been dropped in step 0, so what is left to rescue sits where the gene belongs or at a locus nobody claims. A longer call takes the name of the protein it beat. A gene that stands for no reference protein accounts for any placement in its frame: the Bakta call that won its locus in step 3 over a shorter alignment. When no gene carries the protein's name, that call takes it, as long as the alignment spans at least half of the protein; without this the gene came out with no name and the protein was missing from the table. Two calls named this way: in M. bovis, H37Rv's mmpL6 (Rv1557) is the 3' end of a 2,904 bp gene that the TbD1 deletion cut short in H37Rv CDC1551 and M. bovis carry a 4,821 bp DNA methyltransferase gene whose first 506 residues are H37Rv's Rv2024c A remnant names nothing: the RD12 deletion of M. bovis leaves the last 67 of cyp141's (Rv3121) 400 residues at the end of a 321-codon gene that is not cyp141. The pieces of a cut gene are parts of its protein, not copies of it, so each is missing on its own, and one in place does not keep another out. In the CDC1551 draft a 7.5 kb Bakta call displaced the larger piece of Rv0100, which comes back here though the smaller piece is in.",1],[12,"6. Final deduplication","6-final-deduplication","Where one gene ends up at several coordinates, entries fully contained inside a different gene are dropped, always keeping at least one entry per gene so nothing disappears entirely. After the stop codon check, one gene per reference protein comes first, and the rest stay only as real copies or pieces. deduplicate_rvs.py ranks the genes that share a reference protein: a non-pseudogene over a pseudogene then the longer one then one Bakta also called at exactly those coordinates then the one nearer where the protein belongs by synteny The first keeps its place. The others stay as further copies of the gene when all of these hold, and anything else is a remnant of the gene and goes: at least 90% as long as the first clear of the copies kept not covered by another reference gene for a pseudogene, also 90% as long as the reference protein, which a piece of a split gene is not A genome can carry a gene more than once, and these copies are what the duplication analysis counts: E. coli O157:H7 Sakai keeps 34 that used to be deleted. The pieces of a cut gene stay as well, when the first gene is one of them and all of these hold: each is marked partial at its cut end no two are open at the same end alone: a gene has one first piece and one last, and any number between that a contig holds whole they are clear of each other together they are at most 110% of the reference gene's length Copies and pieces only miniprot placed are all named after their protein, so later ones get _2, _3 to keep identifiers unique.",1],[12,"7. Stop codon check","7-stop-codon-check","check_cds.py translates every surviving CDS. It is given the reference proteome, which settles what a translation alone cannot. No terminal stop codon. Coordinates are extended to the next in-frame stop, searching 300 bp in the right direction for the strand, or as far as the reference protein says is missing when that is further. If the extension introduces internal stops or finds nothing, the original coordinates are kept and the gene is flagged as a pseudogene. An extension goes no further than the reference protein. A gene may be extended to a stop at most 10 residues past the end of its reference protein: that completes an alignment that stopped short. A frame that runs on further has lost its stop or shifted, and the C-terminus it would add is not the gene's; the gene keeps its extent and is flagged as a pseudogene, with the reason, as PGAP and Bakta treat a frameshift. Often that frame runs on into the next gene, whose stop is then the next one (in M. bovis Rv0590 into Rv0590A and Rv3373 into Rv3374), and extending would annotate that gene twice. Genes a contig cut short are partial, not pseudogenes, and each piece of a cut gene is marked this way, under the gene's name: 3' partial: in a draft assembly, a gene whose frame reaches the end of its contig with no stop, from within 30 bp of it, was cut by the assembly, and runs to the last codon there. 5' partial: a gene that opens within a codon of the contig's edge without a start codon, or with a codon that can start a gene but whose protein begins inside its reference protein. A cut leaves whatever codon it falls before at the edge, and one in six codons inside H37Rv genes is one a gene can start with. Reference pseudogenes keep their extent. When the reference protein spans its annotated stretch codon for codon with no stop (PE21, Rv2099c; PPE48, Rv3022c) and the gene ends where it does, the reference annotates it without a stop, and the gene is left as it is and flagged. A reference pseudogene annotated with a partial final codon (oxyR', Rv2427A) gets that base back, since an alignment only covers whole codons. Clipped N-termini are restored. When a gene ends in a stop but is short of its reference protein at the start, and the codons upstream open with a start codon and translate to the missing residues, the gene is extended to cover them. A gene that already starts where its reference does is shorter somewhere inside, and is left alone. Internal stop codons. Flagged as a pseudogene and kept, with the flag propagated to the master table and the GFF3. *-AMAP.clean.faa excludes them. Untranslatable genes are dropped from the annotation table as well as the FASTA files, so the two never disagree.",1],[12,"Threshold summary","threshold-summary","False-positive overlap: case 1 filtering, 80%. Reference match identity: case 1 filtering, 80%. Bakta length advantage: single overlaps, 550 bp. Bakta length advantage: multiple overlaps, 500 bp. Overlap for deduplication: on insert, 70%. Length margin for displacement: on insert, 550 bp. Distance from the predicted position: paralog placements, 10 kb. Proteins placed once, each side, for the local offset: paralog placements, 5. Cover to claim another protein's placement: paralog placements, 50%. Shortest protein looked for exactly: small proteins, 8 residues. Longest protein looked for with substitutions: small proteins, 50 residues. Residues after the first that may differ: small proteins, 20%. Distance from a contig's edge for a placement, or a piece, to be cut there: pieces of cut genes, 30 bp. Shortest part of a protein looked for at the other contig ends: pieces of cut genes, 8 residues. Cover for a gene to account for a placement: rescuing missing genes, 50%. Share of its protein an alignment spans to name a longer call: rescuing missing genes, 50%. Length of a further copy, next to the first: final deduplication, 90%. Cover by another reference gene that rules a copy out: final deduplication, 50%. Length of a gene's pieces together, next to the reference gene: final deduplication, 110%. Stop codon search: stop codon check, 300 bp, or what the reference protein is missing. Residues an extension may run past the end of the reference protein: stop codon check, 10. Distance from a contig end within which a gene counts as cut: stop codon check, 30 bp at the 3' end, a codon at the 5' end. Identity to restore a clipped N-terminus: stop codon check, 90%. These are constants in bin/classify/, tuned on the Mycobacterium tuberculosis complex (MTBC). Another organism may want different values, particularly the 550 bp length margins, which assume genes of roughly bacterial size.",1],[12,"Next steps","next-steps","The workflow: where this sits in the pipeline Outputs: the case tables that record every decision",1],[13,"","","AMAP looks at gene duplication twice: within each genome, and across the whole set.",0],[13,"How the families are found","how-the-families-are-found","Each corrected proteome is searched against itself with Diamond, excluding self-hits. A pair is kept when the identical residues of the alignment cover 80% or more of either the query or the subject, so a short domain match between two otherwise unrelated proteins is not counted as duplication. Kept pairs become links in a graph, and its connected pieces become the duplication families, listed in DUPLICATIONS/{lineage}{sampleId}/duplications{lineage}_{sampleId}_groups.tsv. Because the rule takes the larger of the two shares, a fragment joins the family of the gene it came from: a match that covers almost all of a short protein counts, however little of the long one it covers.",1],[13,"How the communities are built","how-the-communities-are-built","The proteins of every sample's families are then pooled and searched all against all. Within each genome, the pairs that pass the same rule make a network, which the Louvain method splits into communities, with a fixed seed so that a rerun splits it the same way. Two communities are homologous when at least one hit joins them, and each homologous pair is one row of DUPLICATIONS/homologous_communities.tsv, with an interactive network in the matching HTML file.",1],[13,"The columns","the-columns","A row describes two communities: the columns ending in 1 are the first, those ending in 2 the second. The first is the one whose sample comes first by name, or the one with the lower number when both are in the same sample, and the rows come in that order too, so a rerun writes the same file. For each: Genome, Community: the sample, and the community's number in it. Size: how many proteins it holds. Elements: the eggNOG COG categories of its proteins, with how many have each, the largest first. A protein eggNOG gave no category counts as -, eggNOG's mark for none. AvgDegree: links per protein, on average. In a community of n proteins it reaches n - 1 when every protein is linked to every other. Density: the links it has as a share of the pairs it could have. 1 means every protein matches every other. AvgClustering: for each protein, how often two of its partners are linked to each other as well, averaged over the community. 1 when every protein is linked to every other, 0 for a chain or a star. AvgPathLength: how many links separate two of its proteins, on average, over its largest connected piece. Nodes: the proteins themselves.",1],[13,"Reading them together","reading-them-together","Density and clustering at 1 mean copies that all match each other: a recent expansion of near-identical copies. In CDC1551 these are the transposases of insertion sequences, the ESAT-6-like Esx proteins and PE proteins. Lower density and longer paths mean copies that have drifted apart, each still matching some of the others but not all, as M. bovis's copies of the transposases above have. Or, less happily, an artefact of assembly fragmentation, where one gene broken across contigs looks like several partial copies. Cross-checking against the contig count in *-AMAP.qc.tsv tells the two apart. Duplications and assembly quality Repetitive families are exactly what short-read assemblers collapse or fragment. A genome with markedly fewer duplications than its relatives may have collapsed them rather than lost them. The metrics describe the assembly you provided, not necessarily the organism.",1],[13,"Next steps","next-steps","Outputs: where the duplication files live The workflow: where this step sits",1],[13,"Duplications and assembly quality","note-duplications-and-assembly-quality","Repetitive families are exactly what short-read assemblers collapse or fragment.",1],[14,"","","Every tool AMAP calls lives in one image, paururo/amap, published on Docker Hub. Both the docker and singularity profiles use it, so nothing needs installing beyond a container engine. Nextflow pulls the image on first use and caches it. Singularity converts it to an image file of its own on the way, which takes a few minutes once per cache folder: see Singularity on a cluster for keeping one.",0,"nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv singularity"],[14,"What is inside","what-is-inside","Tools: Bakta 1.12.0, miniprot 0.18, Diamond 2.1.24, eggNOG-mapper 2.1.13, pangene 1.1, pathotypr 1.0.0, gfatools. Runtimes: Python 3.9 with pandas, NumPy, Biopython, matplotlib, plotly, NetworkX, intervaltree, pyvis and python-louvain. R 4.4 with the report packages. Bundled data under /reference/, so a standard run on the Mycobacterium tuberculosis complex (MTBC) needs no downloads: MTB_ancestor_reference.fasta: MTBC ancestral reference genome. aa_H37Rv.translated.fas: H37Rv reference proteome. bad.bakta.translated.fas: Curated Bakta false positives. new_refH37Rv.fasta: H37Rv reference genome. sequence.gb: H37Rv GenBank, for Bakta. Mycobacterium_tuberculosis_H37Rv_txt_v4.txt: Mycobrowser functional categories. snps_lineages.tsv: pathotypr lineage markers. Plus the Bakta light database at /data/bakta_db. The eggNOG database is not bundled, because of its size. AMAP downloads and builds it on first use and caches it at --eggnog_db.",1],[14,"Platform","platform","The image is linux/amd64 only, and every Dockerfile stage pins it. On Apple Silicon it runs under emulation, which works but is slow. For anything past a couple of genomes, use an amd64 machine or a cluster.",1],[14,"Pinning a version","pinning-a-version","A tag can move: latest is whatever was pushed last. For a reproducible analysis, pin the image by its digest, and the pipeline's code with -r, since a pinned image under newer code is still a different run: That digest is the image behind the runs on these pages; docker image inspect paururo/amap:latest --format '{{index .RepoDigests 0}}' prints the one you have. Pinning the pipeline has the rest. results/pipeline_info/software_versions.yml records the exact version of every tool a run used, which is what belongs in a methods section.",1,"nextflow run PathoGenOmics-Lab/AMAP -r 947a99dee49d2a2429e014296ac35ba08e911108 -profile docker --tsv samples.tsv -process.container 'paururo/amap@sha256:08bd17311ab323317ceecba6e5bc639a29f4c54c9d95f9a7c48ba80a9fc640ee'"],[14,"Building it yourself","building-it-yourself","Only needed if you are changing what is in the image. The build takes about an hour: it downloads the Bakta database and solves a conda environment with R. Check the result before using it: That verifies the tools, the Python and R libraries, the bundled reference data, and runs the repository test suite with the image interpreter. The last part matters, since a library update can break the pipeline scripts without the image looking any different. Publishing is automated: the Docker image GitHub Actions workflow builds, smoke tests and pushes on a v* tag or on demand. Reference downloads can expire Several reference files are fetched from Google Drive during the build. Those links have expired before. If a build fails on a wget in the data stage, that is why, and the URL needs replacing. The Zenodo-hosted files (MTB_ancestor_reference.fasta, the pathotypr markers) are not affected.",1,"git clone https://github.com/PathoGenOmics-Lab/AMAP cd AMAP docker build --platform linux/amd64 -f docker/Dockerfile -t amap:dev docker/smoke_test.sh"],[14,"Singularity on a cluster","singularity-on-a-cluster","The singularity profile keeps the converted image in /singularity_cache, so each results folder gets a copy of its own, and that setting wins over NXF_SINGULARITY_CACHEDIR: the variable alone changes nothing. To keep one copy for every run, set the folder in a config and pass it with -c: With Nextflow 24.04 and 26.10 alike, nextflow -c shared.config config PathoGenOmics-Lab/AMAP -profile singularity then shows cacheDir = '/shared/singularity_cache', where without it the profile's results/singularity_cache stays, whatever the variable says. The pipeline has to be pulled first: any run does it, or nextflow pull PathoGenOmics-Lab/AMAP. More cluster specifics in HPC clusters.",1,"singularity.cacheDir '/shared/singularity_cache' nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm -c shared.config --tsv samples.tsv"],[14,"Next steps","next-steps","HPC clusters: scratch, shared filesystems, schedulers Troubleshooting: when a run fails",1],[14,"Reference downloads can expire","note-reference-downloads-can-expire","Several reference files are fetched from Google Drive during the build.",1],[15,"","","Past about twenty genomes, a scheduler is worth the setup. Combine a container profile with an executor profile:",0,"nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv"],[15,"The executor profiles","the-executor-profiles","slurm: SLURM, 100 per minute. sge: Sun Grid Engine, Univa, 20 per minute. pbs: PBS, Torque, 20 per minute. lsf: IBM Spectrum LSF, 20 per minute. All four queue up to 200 jobs at once, default to 48 CPUs and 128 GB per job, and enable node-local scratch. SGE additionally requests memory through -l h_vmem, and uses the smp parallel environment. Tune the concurrency to your site's policy if 200 is too aggressive. The queue size belongs to the executor, not to a process, so -process.queueSize changes nothing; set it in a config passed with -c:",1,"executor.queueSize 50 nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm -c queue.config --tsv samples.tsv"],[15,"Fitting your site","fitting-your-site","-process.queue sets the partition or queue, and -process.clusterOptions passes anything else straight through, which is where accounting flags, QoS and node constraints go. On SLURM, Bakta's jobs, the longest a run sends, also ask for --qos=medium. The slurm profile sets that for Bakta alone, and for Bakta it takes the place of -process.clusterOptions, so give Bakta your site's options in a config of your own, with --qos=medium if your cluster has that QoS and without it if not: Then pass it with -c, next to the options for every other job:",1,"nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv --max_cpus 64 --max_memory 256.GB -process.queue long -process.clusterOptions '--account=mygroup' process withName: 'BAKTA_ANNOTATOR' clusterOptions '--qos=medium --account=mygroup' -c bakta.config"],[15,"Scratch","scratch","--scratch is on by default in every executor profile, and it matters. Without it, every process reads and writes directly on the shared filesystem, and Bakta and eggNOG in particular generate enough metadata traffic to slow a cluster down for everyone. With it, work happens on node-local disk and only the results are copied back. Turn it off only if your nodes have no local disk:",1,"nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv --scratch false"],[15,"Where to put things","where-to-put-things","The Singularity cache. The singularity profile keeps the converted image in /singularity_cache, so every results folder gets a copy of its own, and that setting wins over NXF_SINGULARITY_CACHEDIR: the variable alone changes nothing. Keep one shared copy with a config passed with -c: The eggNOG database. Large, built once, reused forever. Point it at persistent shared storage rather than somewhere per-run: The work directory. It gets large, tens of gigabytes for a big run. Put it on a filesystem with room and a generous quota: Keep it until you are sure the run is finished, since -resume needs it.",1,"singularity.cacheDir '/shared/singularity_cache' nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm -c shared.config --tsv samples.tsv --eggnog_db /shared/databases/eggnog -work-dir /scratch/$USER/amap_work"],[15,"Launching","launching","Nextflow's own process must survive the whole run, which will outlive an SSH session. Either run it under tmux or screen, or submit it as a small job of its own: The launcher itself needs very little: it submits and waits while the real work happens in the jobs it spawns. Give it plenty of wall time, though, since it has to outlast everything it submits.",1,"#!/bin/bash #SBATCH --job-name=amap --cpus-per-task=2 --mem=8G --time=48:00:00 --output=amap_%j.log module load singularity nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm -c shared.config --tsv samples.tsv --eggnog_db /shared/databases/eggnog -work-dir /scratch/$USER/amap_work -resume"],[15,"Memory and retries","memory-and-retries","A process killed by the out-of-memory killer is resubmitted with more memory, scaling with the attempt number up to --max_memory. That handles the occasional unusually large genome without you having to size everything for the worst case. A task run per sample that fails for another reason is tried twice more and then set aside, so the run finishes without that sample and says so under Ignored: see a sample is missing. eggNOG is the memory-hungry step. It runs once over all samples rather than once per sample, which keeps a shared filesystem from being hammered but does mean one large job. The highmem profile exists for this:",1,"nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm,highmem --tsv samples.tsv"],[15,"Watching a run","watching-a-run","squeue -u $USER shows the jobs. When it finishes, pipeline_info/ holds the execution report, the timeline, and a trace with per-process CPU, memory and runtime, which is what to look at before adjusting resources for the next run.",1],[15,"Next steps","next-steps","Troubleshooting: when something fails Containers: what the image holds",1],[16,"","","Each problem below is a heading, with the message the pipeline or the system prints in quotes where there is one, so you can search for what you see.",0],[16,"Reading a failure","reading-a-failure","When a process fails, Nextflow prints the work directory for the failed task. That directory has everything you need: .command.err is almost always where the answer is. Re-running .command.sh inside the container reproduces the failure by hand. results/pipeline_info/execution_trace_*.txt shows every task with its status, CPU, memory and runtime, which is the fastest way to see whether a failure was resources or logic.",1,"work/ab/cd1234.../ .command.sh the exact script that ran .command.log everything it printed .command.err stderr on its own .exitcode what exited with"],[16,"Before the pipeline starts","before-the-pipeline-starts","",1],[16,"\"Unexpected input: '('\" in nextflow.config","unexpected-input--in-nextflowconfig","Nextflow 25.10 and later read pipelines with a stricter parser, which older copies of AMAP do not pass. Nextflow keeps the copy it downloaded first, so fetch the current one with nextflow pull PathoGenOmics-Lab/AMAP and run again. To run an older AMAP release as it is, export NXF_SYNTAX_PARSER=v1 first.",1],[16,"\"Input TSV file not found\"","input-tsv-file-not-found","The path in --tsv is wrong, or relative to somewhere other than where you launched Nextflow.",1],[16,"\"Invalid TSV format\"","invalid-tsv-format","The header must be exactly sampleId and fasta, separated by a real tab. Spaces are the usual culprit. cat -t samples.tsv shows tabs as ^I, and works on both Linux and macOS.",1],[16,"\"FASTA file not found\"","fasta-file-not-found","A path in the sample sheet does not resolve. Absolute paths avoid the ambiguity.",1],[16,"\"not found, neither on the host nor inside the container\"","not-found-neither-on-the-host-nor-inside-the-container","A reference file the run reads is in neither place: a typo in its path, or one of the defaults, which ship in the container, on a run without one. Give the path to your own copy with its reference argument.",1],[16,"Validation failures","validation-failures","A check that fails stops the run, and Nextflow prints the validator's whole report, which check failed and for which sample, under Command output, followed by the task's Work dir. Only a check that passes writes results/pipeline_info/validation.log; after a failure the log is in that work folder. With --strict_validation, a warning fails the same way. Thresholds are in preparing inputs.",1],[16,"Container problems","container-problems","",1],[16,"\"Cannot connect to the Docker daemon\"","cannot-connect-to-the-docker-daemon","Docker is installed but not running: start Docker Desktop, or the Docker service on Linux. docker info answers once it is up, where docker --version answers either way.",1],[16,"\"permission denied while trying to connect to the Docker daemon socket\"","permission-denied-while-trying-to-connect-to-the-docker-daemon-socket","Your user is not in the docker group. Either add it, or use Singularity.",1],[16,"The image will not pull","the-image-will-not-pull","Check network access from the compute nodes, which often differs from the login node. Pull the image once on the login node, into a folder every node sees, and have the runs look there. The singularity profile keeps its images in /singularity_cache, and that setting wins over NXF_SINGULARITY_CACHEDIR, so the variable alone changes nothing; set the folder in a config passed with -c instead: paururo-amap-latest.img is the name Nextflow gives that image in its cache, so the run finds it there and pulls nothing.",1,"singularity.cacheDir '/shared/singularity_cache' singularity pull /shared/singularity_cache/paururo-amap-latest.img docker://paururo/amap:latest nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm -c shared.config --tsv samples.tsv"],[16,"Slow on Apple Silicon","slow-on-apple-silicon","The image is amd64 only, so it emulates. Fine for the test profile, painful beyond that. Use an amd64 machine.",1],[16,"Memory","memory","",1],[16,"Killed with exit 137, 140 or 143","killed-with-exit-137-140-or-143","Exit 137 is the out-of-memory killer, 140 a time limit and 143 a SIGTERM, the way a scheduler usually ends a job at its limits. AMAP retries a task that exits with any of them, asking for more memory on each attempt, so a single occurrence in the log that later succeeds is normal. Persistent failures mean raising the ceiling, with --max_memory 256.GB for instance. On a cluster, that is the whole answer. On a macOS or Windows laptop it is the wrong one: Docker Desktop runs containers inside a VM with its own memory ceiling, commonly 8 GB, and that ceiling is what killed the process. Nextflow sized the request against the host and never saw it. Raising --max_memory only makes the request larger than a limit it was already over. Raise the VM in Docker Desktop under Settings, Resources, Memory, then keep --max_memory at or below it.",1],[16,"eggNOG runs out of memory","eggnog-runs-out-of-memory","It is the hungriest step, requesting 50 GB, and it runs once over all samples together. On a machine that cannot reach that, --skip_functional_annotation drops it along with the pangenome, the duplication analysis and final_report.html, and still gives you the corrected genes and amap_report.html. On a cluster, use the highmem profile, or point it at a large-memory partition:",1,"nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm,highmem --tsv samples.tsv"],[16,"Filesystem","filesystem","",1],[16,"\"No space left on device\"","no-space-left-on-device","Usually the work directory, which grows to tens of gigabytes on a large run. Move it somewhere with room by adding -work-dir /scratch/$USER/amap_work to the command. Also check your $TMPDIR quota, since Diamond and Bakta both use it heavily, and the folder --eggnog_db names: eggnog.db alone is 41.4 GB.",1],[16,"Everything is slow on a shared filesystem","everything-is-slow-on-a-shared-filesystem","Confirm --scratch is on. It defaults to true in the executor profiles, but a standard or docker profile on a cluster will not have it.",1],[16,"Cache misses on resume","cache-misses-on-resume","NFS timestamp drift across nodes invalidates Nextflow's cache. The slurm profile already sets process.cache = 'lenient' for this. With another executor, add -process.cache lenient to the command.",1],[16,"The eggNOG database","the-eggnog-database","",1],[16,"It downloads every run","it-downloads-every-run","--eggnog_db is pointing somewhere per-run. Give it persistent shared storage.",1],[16,"It rebuilds itself unexpectedly","it-rebuilds-itself-unexpectedly","A bacteria.dmnd under 100 MB is treated as a truncated download and rebuilt. Usually means an earlier download was interrupted.",1],[16,"\"has eggnog_db.ok, so EGGNOG_DB will not run again\"","has-eggnog_dbok-so-eggnog_db-will-not-run-again","eggnog_db.ok tells Nextflow the database is built, so it never runs EGGNOG_DB again, but what it needs is not all there. Delete the files the message names and the next run builds it again.",1],[16,"Results that look wrong","results-that-look-wrong","",1],[16,"Far fewer genes than expected","far-fewer-genes-than-expected","Check the assembly first: *-AMAP.qc.tsv has the contig count and the N50. A gene that a contig end cuts is not lost: it comes back in pieces, each under the gene's name and marked partial at its open end: partial in the master table, the Partial state in amap_report.html. Only a piece under 8 residues is too short to place: in CDC1551 cut into 40 contigs, 22 of the 25 smaller pieces were placed, and the other three were five or six codons long (pieces of cut genes). *-rvmissing.tsv is a different list: the reference genes Bakta missed and AMAP put back.",1],[16,"\"genes lost to (Start, End, Strand, Contig) collisions\"","genes-lost-to-start-end-strand-contig-collisions","Two of the classification's sources listed a gene at exactly the same place, and one entry was kept. Genes on different contigs never collide, since the contig is part of the key.",1],[16,"A sample is missing from the results (\"Ignored : 1\")","a-sample-is-missing-from-the-results-ignored--1","A task run per sample that fails for any reason but its limits (exit 137, 140 or 143) is tried twice more and then set aside, so one bad genome does not stop the others. The run still ends โœ“ COMPLETED, its summary counts the task under Ignored, and a warning after it says how many failed. The genome's results stop at that step: set aside at the correction or before it, it has no folder under AMAP/, and if it was the correction itself, amap_report.html names it at the top. To find which task it was and why: The trace's second column is the task's hash, the start of its folder under work/, and that folder's .command.err says why. A dot in a sample ID is one cause: see preparing inputs.",1,"grep -w FAILED results/pipeline_info/execution_trace_*.txt"],[16,"The clean proteome has fewer sequences than the GFF3 has genes","the-clean-proteome-has-fewer-sequences-than-the-gff3-has-genes","Expected. *-AMAP.clean.faa drops pseudogenes and internal-stop sequences. Use *-AMAP.faa if you need one protein per annotated gene.",1],[16,"Almost everything is case 1","almost-everything-is-case-1","Your samples are too far from the reference for the alignments to find anything, so there is nothing to correct against. Use a closer reference. See custom reference.",1],[16,"The Mycobrowser sections are empty","the-mycobrowser-sections-are-empty","Expected with --skip_mycobrowser, or when the Mycobrowser file is missing. The pipeline warns and continues.",1],[16,"Still stuck","still-stuck","Open an issue at github.com/PathoGenOmics-Lab/AMAP/issues with: the command you ran the failing task's .command.err results/pipeline_info/software_versions.yml results/pipeline_info/params.yml The last two pin down the exact versions and parameters, which is usually what the answer turns on.",1],[17,"","","What changed in each version.",0],[17,"On main, not yet released","unreleased","Annotating H37Rv against its own reference now gives back every CDS of the AL123456.3 GenBank record at exactly its coordinates, strand and name: 4031 of 4031, against 4020 before. The changes apply to every genome, so results are not directly comparable with 1.0.1.",3],[17,"On main / Correctness","unreleased-correctness","Paralog copies no longer take another gene's place: a protein with a paralog was placed at both loci, and which copy survived came down to insertion order. On H37Rv itself that lost Rv1150, Rv1945 and Rv2828c; on M. bovis it reported genes the strain lacks (the mce3 operon of RD7, the phiRv2 prophage of RD11, plcBC of RD5) as copies sitting on their paralogs. Each placement is now checked against where its protein belongs, predicted from the sample's own offset to the reference so that indels do not throw it off Strand of a resolved conflict: taken from the protein's first placement anywhere in the genome instead of the one being resolved, so esxM (Rv1792), Rv3466 and two IS6110 copies came out at their own coordinates on the wrong strand; a Bakta call chosen over an alignment also took the alignment's strand Reference pseudogenes keep their extent: PE21 (Rv2099c) and PPE48 (Rv3022c), which the reference annotates without a stop, were extended to the next one, and oxyR' (Rv2427A) and Rv3216 lost the partial final codon the reference gives them The 1.0.1 fix for reference pseudogenes never fired: it looked for reference coordinates in a column that has not carried them since March 2026, and its tests wrote them in by hand. check_cds.py now reads the reference proteome instead Alignments that stop short are completed: miniprot can miss a protein's N-terminus (Rv3611 on CP110619) or leave a repetitive C-terminus unaligned further from its stop than the 300 bp searched; the reference protein now says how much is missing Ties between copies of a gene go to the one Bakta also called, then to the one where the protein belongs by synteny, not the one that comes first in the genome (E. coli K-12 b2635) Draft assemblies: genes were compared on coordinates alone, and every contig starts at 1, so cut into 40 contigs CDC1551 had 87% of its proteins translated from the wrong contig. Every gene is now matched, translated and written on its own contig, and Bakta runs with --keep-contig-headers so that its contig names match the FASTA's Case 4 is grouped strand by strand: about 8% of its groups (101 of 1255 on H37Rv) put an alignment on one strand together with a Bakta call on the other A complete alignment is not displaced by a longer call in its frame: such a call ends at the same stop and only starts further upstream (plcD on CDC1551). Nor does a call that spans several reference genes win against them (Rv3022c and Rv3021c, one reading frame in CDC1551) Reference genes are protected again: the check looked for coordinates in identifiers that no longer carry them, so any call of similar length could displace a reference gene (Rv1614) Missing genes are recovered by what covers them: genes stored under Bakta locus tags looked missing by name, so a placement a codon off its gene went back in as a second copy. With the case 4 changes, ORFs annotated twice fall from 10 to 3 on CDC1551 and from 16 to 6 on M. bovis, where the reference has none Real copies of a gene are kept: deduplicate_rvs.py deleted every copy of a gene but one, 34 genes on E. coli O157:H7 Sakai, which were then missing from the annotation and from the duplication analysis. Further copies get identifiers of their own Genes that lost their stop no longer swallow the next gene: check_cds.py extended a gene without a stop to the next one in its frame, which after an indel is the next gene's, so that gene was annotated a second time in the same frame (M. bovis Rv0590 over Rv0590A, Rv3373 over Rv3374, Rv2293c over Rv2292c). Such a gene now keeps its extent as a pseudogene, unless the extension only restores residues its reference protein has. ORFs annotated twice fall from 3 to 2 on CDC1551 and from 6 to 3 on M. bovis, and none of those left comes from an extension Small proteins: miniprot placed none of the 33 proteins of 8 to 29 residues in E. coli K-12, and put the toxin IbsC (b4665) only on its near-identical neighbour IbsD. Every protein without an exact placement is now looked for in the six-frame translation of the genome, which takes K-12 against itself from 4252 to all 4286 of its genes, its alternative-start isoforms counted with their gene, and Sakai from 198 to 220 of 220 Frameshifts and lost stops: a gene was extended to the next stop in its frame however far that was, and came out as a complete gene with a C-terminus its reference lacks; 18 in CDC1551 and 20 in M. bovis. It now keeps its extent and is flagged as a pseudogene when that stop lies more than ten residues past the end of the reference protein, as PGAP and Bakta treat a frameshift, which also covers a frame that runs into the next gene Genes a contig end cuts: in a draft assembly they came out as pseudogenes, or as complete genes, without a word. A gene whose frame runs off the contig, or that opens at its edge without a start codon, is now marked partial at that end. In CDC1551 cut into 40 contigs, the larger piece of each of the 25 genes a cut runs through is marked so The other pieces of a cut gene: miniprot reports only the placements within 5% of a protein's best, so only the larger piece of a gene a contig end cuts was annotated. What that placement lacks is now aligned again as a protein of its own, and its placement at the end of another contig is annotated under the gene's name and marked partial: 47 of the 50 pieces in the CDC1551 draft instead of 25, the other three being five or six codons long. A gene that opens at a contig's edge with a codon that can also start one, as one in six codons inside H37Rv genes can, is marked partial too when its protein begins inside its reference protein Disrupted copies of a gene: deduplication kept further copies only when intact, so a whole copy that is a pseudogene was deleted with the pieces of split genes; it now stays when nearly as long as the reference protein GFF3 references: each CDS line was translated on its own, so a CDS of several lines came out in fragments, and proteins under 11 residues were dropped to hide them. The lines are joined per CDS and translated with the bacterial code, reading transl_except, and E. coli K-12's GFF3 gives the same 4318 proteins as its GenBank record, 24 of which were missing Genes longer than their reference keep its name: a Bakta call that won its locus over a shorter alignment in its frame kept no name, and the protein was missing from the table: Rv2024c in CDC1551 and M. bovis, mmpL6 (Rv1557, whole in M. bovis where TbD1 cut it in H37Rv), Rv1068c, Rv1917c and Rv3135 in M. bovis, fliD (b1923) and b0282 in Sakai. The call now takes the name when the alignment spans at least half of the protein, so a remnant does not name the gene it ends (the 67 residues of cyp141 that RD12 leaves in M. bovis) Small proteins with a few substitutions: only exact copies were looked for, so Sakai's ten K-12 micro-proteins with one to four residues changed were not annotated. A protein of up to 50 residues with no exact copy is now looked for with up to a fifth of its residues changed; a stop codon in place of a residue counts, and the gene is then a pseudogene (b1229) Synteny no longer hangs on the order of the rows: the anchors were sorted by start and searched by midpoint, so two anchors starting together sent a placement to whichever window the row order gave, and adding unrelated rows moved Sakai's b2127 24 kb and handed its locus to the paralog b1162. A locus several proteins are placed at exactly no longer anchors either: the single IS6110 of M. bovis made 34 anchors with reference positions all over the genome, and came out under all 17 IS6110 names of each of its genes; they are now Rv2814c and Rv2815c, its own Genes without a COG category are counted once: count_annot.py filled in a missing category as Unassigned and split it into letters, as it splits JK into J and K, so every gene eggNOG gave no category was counted once in A, D, E, G, I and U and twice in N and S, in *-AMAP.COG.tsv and in the report's COG figures and statistics, where it also shrank every other category's share. A test-profile run with a stand-in for eggNOG that gave none of its 4036 genes a category reported 4036 genes in each of those six and 8072 in N and S. Such a gene now counts once, as NaN, which the report shows as None, and its percentage is of all the genome's genes like every other category's COG categories R, W, X, Y and Z in the report: its COG figures took their categories from a list that ran from A to V without R, so a gene eggNOG put in R, W, X, Y or Z was plotted as NA in the overview and in none of the per-category figures, which stayed empty for those five, X - Mobilome (prophages and transposons) among them. The categories now come from the report's own list of figures, A to Z and None, so each is plotted in its own. The statistics table never used that list and is unchanged The QC table counts the pseudogenes AMAP called: annotation_qc.py read the first of Is_Pseudogene, pseudogene and is_pseudogene that the master table has, and every master table has Mycobrowser's Is_Pseudogene, empty when Mycobrowser is off. So pseudogene_count in -AMAP.qc.tsv and the report's QC table was 0 in every organism-agnostic run (*E. coli Sakai against K-12, where AMAP called 129 pseudogenes), and in an MTBC run the number of the sample's genes named after one of the 13 that Mycobrowser marks as pseudogenes in H37Rv, whatever AMAP made of them: 13 for CDC1551, where AMAP calls 123 and finds 6 of those 13 intact. It now counts AMAP's own call, the master table's pseudogene column, the same genes the GFF3 marks pseudo=true and the GenBank /pseudo The report's pseudogene chart counts AMAP's calls, sample by sample: its Pseudogene Detection section counted the master table's genes flagged in Mycobrowser's Is_Pseudogene or in AMAP's pseudogene column, lineage by lineage, as \"Pseudogenes per Lineage (Mycobrowser annotation)\". Is_Pseudogene says which genes are pseudogenes in H37Rv, not in the sample, so CDC1551 came out at 129 where AMAP calls 123 and the QC table says 123, the other 6 being genes Mycobrowser marks that CDC1551 has intact (Rv0947c, Rv1150, Rv2023A, Rv2098c, Rv3021c, Rv3324A), and without Mycobrowser the chart showed AMAP's calls under Mycobrowser's name. It also left out a lineage with none, and added up a lineage's samples, so a lineage with more samples looked as if it had more pseudogenes. REPORT_INPUTS now hands the report every master table row with the sample it is from, and the chart shows how many pseudogenes AMAP called in each sample, by lineage, a sample with none at 0, next to a table of them. The master table explorer shows the note that says why a gene is a pseudogene A sample with no gene in a category counts 0 in the report: count_annot.py writes a row for each COG or Mycobrowser category a sample has genes in, and the report plotted the rows as they came, so a sample with no gene in a category was left out of that category's figures, of the Dunn tests between lineages run on them, of the overview and of the statistics table, instead of counting 0. Its lineage's box, mean and median were those of the samples that have some, a lineage none of whose samples has any dropped out of the comparison, and the comparison with the median never showed a sample that had lost a category. Each sample now counts 0 in every category another sample of the run has genes in, and the statistics table's Count is every sample of the lineage. A category no sample has keeps its empty figure, and *-AMAP.COG.tsv and *-AMAP.MycoBrowser.tsv are unchanged The clean proteome leaves out the pseudogenes AMAP called: export_clean_proteome.py took them from the first of Is_Pseudogene, pseudogene and is_pseudogene that the master table has, which is Mycobrowser's, and looked each protein up by its whole FASTA ID, L4_CDC1551_Rv0001;+;0-1524 where the master table has L4_CDC1551_Rv0001, so it never found one and its log said 0 pseudogenes. Only the proteins with an internal stop codon were left out, and the pseudogenes without one stayed in -AMAP.clean.faa, which the docs say has none: 42 of the 123 AMAP calls in CDC1551 (a frameshift or lost stop codon, no stop codon within reach, no stop codon as in the reference) and 27 of the 129 in *E. coli Sakai against K-12. It now leaves out the genes whose pseudogene is yes, found by the ID before the ;, and on both genomes holds the CDS the GFF3 does not mark pseudo=true, 3891 and 5194, with the proteins the GenBank translates for them. A master table without that column stops it, where a warning let every pseudogene through",3],[17,"On main / Pipeline","unreleased-pipeline","Pseudogenes in the GFF3 and GenBank: the GFF3 never said which genes were pseudogenes, and the GenBank only for genes with an internal stop, though the master table knew. Both now carry AMAP's flag with its reason (pseudo=true and Note, /pseudo and /note), and mark the ends a contig cut off (partial=true with start_range or end_range, < or >); the master table and *-AMAP.tsv have them as their last columns The default H37Rv reference under Docker: its files live inside the container and were handed to processes as paths, so Nextflow mounted /reference from the host, which Docker Desktop refuses and Docker on Linux mounts empty over the container's own. BUNDLED_REFERENCE copies them out first -profile test: VALIDATE_INPUTS still required the sample sheet the test profile does without. 1.0.1 had only been checked with -preview, which runs no task -resume kept results made by older code: Nextflow hashes a task's inputs and script, not the files it runs by path, so after an edit to the classifier or any script under bin/, -resume reused the old results. Every process that runs AMAP code now lists those files in ext code, which puts them in its cache key; edit bin/classify/cases.py and TYPE_CASES and FIX_CDS run again while PATHOTYPR and Bakta stay cached PATHOTYPR no longer depends on the classifier: its script said pathotypr classify, and Nextflow adds any bin/ entry a script names as a bare word to the cache key, so a change in bin/classify/ re-ran PATHOTYPR, Bakta and everything after them. The subcommand is quoted Nextflow 25.10 and later: the pipeline stopped before running anything, because the strict parser those versions use by default rejects the helper functions in nextflow.config, the statements outside the workflow in main.nf, the bare env(SAMPLE_NAME) output and publishDir paths that name a task input. Nextflow 26.10 also refuses the return that ended a --validate_only run. The resource caps are now the resourceLimits directive, which still takes --max_cpus, --max_memory and --max_time from the command line, a profile or a -c config. Nextflow 24.04, where that directive first appeared, is the new minimum --skip_mycobrowser in organism-agnostic mode: a custom reference from another genus turned it on by reassigning the parameter, which current Nextflow ignores with a warning, so the run went ahead with Mycobrowser enabled. It is now worked out in the parameter's default Tasks leave the pipeline untouched: the first time a copy of AMAP ran, Python cached the bytecode of AMAP's own modules inside it, in bin/classify/_pycache_, and with pathotypr classify naming that directory in PATHOTYPR's script, the first -resume re-ran PATHOTYPR, Bakta and everything after them. Tasks now run with PYTHONDONTWRITEBYTECODE=1 Samples with no Bakta call outside cases 2 and 3: when every Bakta call had an alignment at exactly its coordinates, which a small input can do, TYPE_CASES stopped with a KeyError, because the empty column of overlaps was read as a list of column names. Case 1 and case 4 now come out empty and the rest is classified as usual. An empty miniprot table stopped it the same way Samples Bakta calls no CDS on: with no contig names from Bakta to compare, the contig check took them for different names and stopped the run, and a Bakta table with nothing but its header did not read. The genes miniprot places now come out as missing from Bakta, like any gene it did not call --version: --help and the parameter docs have listed it since 1.0.0, but nothing read it, so nextflow run PathoGenOmics-Lab/AMAP --version started a run. It now prints the pipeline and Nextflow versions and exits before any task Drafts named after one of their contigs: pathotypr writes a summary row for each contig that carries lineage markers, and PATHOTYPR took the sample's name from the first, so CDC1551 cut into 40 contigs, sample CDC1551d, came out as L4_contig_14, and so did every file after it. Two drafts whose first row named the same contig became one sample, the results of one overwriting the other's. A genome is now named _ from the sample sheet, the lineage of a draft being the one the markers of all its contigs add up to, and the contigs of a draft keep their names instead of the first one taking the sample's. A closed genome whose header is its sample ID comes out as before Headers with a description: the name was the whole header line, so the tutorial's CDC1551 (NC_002755.2 Mycobacterium tuberculosis CDC1551, complete sequence) was named with spaces in it, Bakta failed on it three times and the run finished without it, and without failing. Such a genome now takes its sample ID like any other, and so does the test profile's: L4_H37Rv_test, where it was L4_CP110619_H37Rv An input could be emptied: PATHOTYPR wrote the renamed genome next to the link Nextflow stages its input as, so when the new name was the file's own, sample G26689 in L4_G26689.fasta for instance, the input was overwritten with nothing and the run went on without it. The input is now staged in a directory of its own Sample IDs with underscores: the steps after the correction told a sample's lineage from its sample ID by splitting its name at underscores, though typing only puts the lineage in front and sample IDs may hold underscores of their own. L4_H37Rv_test was counted as genome H37Rv, the tutorial's Beijing_1 and a Beijing_2 as one genome Beijing in the category counts, the report and the duplication network, an untyped H37Rv_test as lineage H37Rv, and each gene of an untyped sample with no underscore as a genome of its own in the network. PATHOTYPR now says which lineage it named a genome after and every step is handed the sample ID and the lineage. The case summary gains a Sample column and names an untyped sample as it is, not Untyped_, and the report no longer reads the first gene not in the reference as a header, nor stops when every gene has one Genomes kept outside the run's folder: VALIDATE_INPUTS staged only the sample sheet and opened the paths written in it from inside its container, which sees only the folder that the work directory and the sheet share. Genomes anywhere else, /data/genomes while running from ~/project for instance, were \"FASTA file not found\" and the run stopped before anything ran, and a relative path, which the sheet may use, was read from the task directory with or without a container. The genomes are now inputs of the process, each in a directory of its own so that two with the same file name do not collide, and each is read where Nextflow staged it Runs with functional annotation stopped at validation: VALIDATE_INPUTS looked for --eggnog_db in its own task directory, where the default eggnog_db, relative to the launch directory, never is, and took a database not built yet for an error, though EGGNOG_DB builds it. It now looks where EGGNOG_DB builds it, staged when it exists, and stops the run only when eggnog_db.ok is there without a complete database, since Nextflow then never runs EGGNOG_DB again A Bakta database on the host: --bakta_db reached the tasks as a path the container does not mount. With -profile docker, validation called the database missing and stopped the run, and where nothing validated it, as under -profile test, every sample failed three times and was ignored, and the run ended with nothing annotated and exit 0. A database on the host is now staged into BAKTA_ANNOTATOR, PREPARE_REFERENCE and VALIDATE_INPUTS like any other input; the default inside the container is used as before A warning on every run: validation expected a db.sqlite3 that no Bakta database has. It now checks for version.json and bakta.db, which Bakta cannot run without, so their absence stops the run, and names the database's type and version --strict_validation: the documentation said validation warnings could be made to stop the run, but nothing passed the script's --strict on. The new parameter does eggNOG's database under Docker: EGGNOG_DB built it in a folder it worked out for itself from --eggnog_db, and having no inputs, its container saw that folder only when Nextflow happened to mount it with the work directory and the pipeline's bin/, through a parent folder they share. Otherwise, as when running from /data/project with the pipeline in ~/.nextflow, the database was downloaded and built inside the container and lost with it, and only eggnog_db.ok reached the host: later runs skipped EGGNOG_DB, and ANNOT_EGGNOGMAP_BATCH found no bacteria.dmnd. A database already there without eggnog_db.ok was built again. main.nf now makes the folder on the host and hands it to EGGNOG_DB as an input. A folder left with only eggnog_db.ok needs that file deleted, as validation says eggNOG's database in a folder of any name: ANNOT_EGGNOGMAP_BATCH asked eggNOG-mapper for eggnog_db/bacteria.dmnd and --data_dir eggnog_db, but Nextflow stages a folder under its own name, so with a --eggnog_db called anything else, as the HPC guide's /shared/databases/eggnog or the tutorial's ~/databases/eggnog, EGGNOG_DB built the database and the annotation stopped at \"DIAMOND database eggnog_db/bacteria.dmnd not present\", with or without a container. The folder is now staged under that one name, whatever it is called. The task's tag and first log line also gave a single sample's FAA size in bytes as the number of samples (batch_1519647_samples under -profile test) Reference files on the host: BAKTA_ANNOTATOR gave Bakta --ref_annotation, or --bakta_proteins, as a path written into its script, which a container does not see, nor a task directory when it is relative, so the organism-agnostic mode failed under -profile docker: with K-12 as the reference for Sakai, Bakta called the \"user proteins file\" not valid three times, the sample was ignored and the run ended with exit 0. Likewise --pathotyper_snps and --pathotyper_ref on the host left every genome untyped, as NA_, --mycobrowser_ref left the Mycobrowser categories out, and one --miniprot_ref_* on the host while the others kept their defaults made BUNDLED_REFERENCE fail, so nothing was corrected. A file on the host is now staged into the tasks that read it, each in a directory of its own, and a default is read inside the container as before Reference files that are nowhere: a typo in one of those paths, or a default on a run without the container, went unnoticed until the samples were ignored or left untyped. Validation now checks each reference file the run reads and that is not on the host, inside the container where the tasks read it, and stops the run when it is not there. A missing Mycobrowser table only warns, since GET_OUTPUTS goes on without it The pangenome without lineage typing: PANGENE aligned each sample's proteins onto the genomes PATHOTYPR renamed, so with --skip_typing, which every custom reference from another genus needs, it had none and wrote empty files for every sample. It now aligns them onto the sample sheet's genomes, each copied to .fasta first, so that two files of the same name (assembly.fasta from two assemblies) stay apart and a .fna is not passed over Genes not in the reference, sample by sample: the report counted only the samples that had some, so a sample whose genes all matched the reference dropped out of its lineage's box instead of counting 0, and when no sample had any (H37Rv against itself, as in -profile test) the section was an empty plot. Every sample is now counted, the section says so when none has any, and each point is drawn at its count, on an axis from 0, where it was jittered up and down by as much as 0.4 The report's pangenome section lost a genome: pangene names each genome #0, and the report read the merged matrix with R's default comment character, so its header stopped at the first #. One genome column was dropped and the gene names became row names, so the core/accessory and frequency plots used a single column and the heatmap never appeared. The missing-gene and duplication tables, which carry the reference annotation's gene IDs as given, are now read without a comment character too Genes across the origin of a circular sequence: Bakta reads a sequence as a complete replicon when its FASTA description says complete (NCBI's \"complete genome\" and \"complete sequence\") or circular=true (Unicycler), joins its ends and calls a gene across its origin from its start to its stop, start after end. TYPE_CASES stopped on such a gene: E. coli O157:H7 Sakai with NCBI's headers, whose stcE runs across the origin of plasmid pO157 (92563..2502), failed three times and was left out, and the run finished without failing. Past it, check_cds.py dropped such a gene as invalid coordinates, deduplication and the QC table gave it a negative length, the GFF3 wrote it backwards, the GenBank writer stopped on it and the Panaroo GFF3 spanned it over the whole sequence. It now goes through whole and is written as Bakta writes it: start after end in the tables, join() across the origin in a circular GenBank record, and in the GFF3 its end plus the sequence's length on a region marked Is_circular=true. A gene near the origin of such a sequence is read across it too, so Rop on plasmid pOSAK1, of which Bakta called the first 141 bp, comes out whole. Annotated against K-12, Sakai with NCBI's headers now has 5331 genes in common with the same genome under bare names, which Bakta reads as linear, and differs in stcE and Rop, whole where the linear reading has 804 of stcE's 886 residues and no Rop, and in two phage genes Bakta starts elsewhere on a complete genome Typed closed genomes were annotated as linear: PATHOTYPR renamed a genome of one sequence to _ and nothing else, so the description Bakta reads a complete, circular replicon from never reached it, whatever the header said, and a gene across the origin of a closed genome came out as two pieces under one name, marked 5' and 3' partial. Only the sequence's ID is replaced now, and the description after it stays, as with --skip_typing. Typed from NCBI's headers, CDC1551, CP110619, AL123456.3 and M. bovis get the same genes as before, in place, name, flags and protein, though Bakta starts a few of its own calls elsewhere on a complete genome (dnaA at 1 instead of 34), so the gene IDs that are Bakta locus tags can be numbered differently. H37Rv turned so that the origin cuts gyrB, gyrA or pknB gives back all 4031 GenBank CDS, where it gave 4030 with the cut gene in two pieces. The GenBank record of such a genome is marked circular, with the description in its definition, and its GFF3 has a region marked Is_circular=true The report's duplication summary: it printed the number of homologous community pairs as markdown from a chunk that did not pass its output on as markdown, so the report showed ## 48 homologous community pairs detected across all genomes. in a code box. It is now a sentence. The example report's duplication table had columns GET_ALLDUP never writes, so its Network Metrics tab said there were none; the demo data now has the columns GET_ALLDUP writes and the example is rendered again Modularity in homologous_communities.tsv: Modularity1 and Modularity2 were computed with the whole community as its only group, which makes them 0 whatever the community, as on all 29 rows of CDC1551 against M. bovis. The two columns are gone COG categories in homologous_communities.tsv: with numpy 2, as in the container, Elements1 and Elements2 read {'L': np.int64(4)}, and they counted each protein eggNOG gave no category in category + or -, since its ID keeps the strand FIX_CDS wrote. They are now plain counts, the largest first and ties in alphabetical order rather than in an order that changed from run to run, and a protein without a category counts as -, eggNOG's mark for none The order of -profile: from Nextflow 25.10, whose strict parser applies the profiles in the order -profile names them and lets the last one win, the laptop caps of the container profiles took over from the others: -profile slurm,singularity held every task of a cluster run to the launching machine's CPUs and 16 GB, where -profile singularity,slurm gave it 48 CPUs and 128 GB, -profile test,docker ran the test with 16 GB and -profile highmem,slurm with 128 GB. No profile sets a cap now: they are set after the profiles, from whether highmem, test or a scheduler profile is on, so every order gives what Nextflow 24.04 gave. The parameter docs gave 24 CPUs and 50 GB as the defaults, which no run got The duplication files are the same from one run to the next: homologous_communities.tsv listed its pairs of communities in the order of a set, which changes with Python's hash seed from one run to the next, and with it which community of a pair came first and the last digit of AvgClustering, an average over the community's proteins that networkx went through in the order of their set: 0.5833333333333333 or 0.5833333333333334 for the same pair of CDC1551 and M. bovis. The pairs now come in the order of the communities, the sample first by name on the left, and the metrics go through the proteins in alphabetical order. Each family in duplications_*_groups.tsv lists its proteins in alphabetical order, where they too came in the order of a set, and each sample's two network drawings no longer start from a random layout and a random partition, so every file both steps write is the same run after run No web browser: both duplication steps wrote their networks with pyvis' show(), which also opens the file in a web browser, so a run without a container on a desktop opened a browser tab for every network. They are written as before, without being opened A custom reference with a gene across the origin, or with several sequences: PREPARE_REFERENCE named each reference protein after where its pieces lie, so tagA on plasmid pO157 of E. coli O157:H7 Sakai, join(92527..92721,1..2502) in NCBI's GenBank record, became pO157p01_1-92721, the whole plasmid, and lost its stop; from NCBI's GFF3, which writes it 92527..95223, it came out as its first 65 of 898 residues. Bakta's stcE on the reference, 92563..2502, matched no gene and was listed among the false positives, and the calls were compared with the genes of every sequence at once: seven calls on pO157 and two on the chromosome counted as matched by genes of the other at the same numbers, and the GenBank's 1-92721 hid two more. With Sakai as its own reference, the sample with NCBI's headers came out with tagA as 1..2502, 65 codons short and not marked partial (GenBank), or from Bakta's later start (GFF3), and with bare headers the GFF3 lost the 2502 bp piece altogether. A gene across the origin now ends past the sequence's last base, as GFF3 writes it (pO157p01_92527-95223, with its stop), each protein names its sequence (contig=), Bakta keeps the FASTA's names on the reference, and each call is compared with the genes of its own sequence, on both sides of the origin; calls on a sequence the annotation has no gene on are not judged. Both formats now give the same 5155 proteins and the same 260 false positives, and the sample with NCBI's headers gets tagA exactly, 92527..2502. K-12's 142 false positives are unchanged pipeline_info/params.yml is the run's own: with -resume it came back from the cache, so it held the first run's parameters whatever later runs were given, since its script reads params as a whole, which Nextflow 26.10's default parser leaves out of the task's hash. And all but the first parameter fell outside params:, with a stray YAML line at the end, because Nextflow strips a script's indentation once the script is filled in. DUMP_PARAMS now never comes from the cache, and writes each value as a JSON string under params: Bakta's SLURM QoS only on SLURM: Bakta's jobs asked for --qos=medium under every scheduler, though only SLURM's sbatch takes it, and under -profile sge it took the place of the memory request the sge profile gives every task. The slurm profile now sets it for Bakta The Mycobrowser table is checked whenever it is read: validation checked --mycobrowser_ref only with the functional annotation, though the interactive report reads it in every run, so under --skip_functional_annotation a wrong path went unwarned and the report came out without Mycobrowser's categories The duplication networks draw offline: both duplication steps loaded their networks' drawing library from a CDN, so without the internet the pages drew nothing. The library now goes inside each page, about 0.7 MB environment.yml pins eggNOG-mapper 2.1.13, the version the container ships, where it said 2.1.12 The pangenome shared no gene between samples: PANGENE aligned each sample's own proteins to the other samples' genomes, ran pangene once per sample and named each column _#0, and REPORT_INPUTS merged those matrices on gene names that carry the sample's name and coordinates (A4_Mbovis_Rv0001;+;0-1524, L4_CDC1551_Rv0001;+;0-1524), so no row was ever in two samples and no gene was ever core. With N samples the matrix had N(N-1) columns: on the first 2 Mb of CDC1551, M. bovis and H37Rv the report showed 5379 gene families in 6 samples, all shell, with 200 genes of one genome as its heatmap, and as no genome was searched for its own genes, the interactive report found no gene in one genome alone. PANGENE now uses pangene as it is meant to be used: one set of proteins, the reference proteome and each sample's genes that no reference gene matches, aligned to every genome, the sample's own included, and one graph for the run, N alignments where there were N(N-1). A row is a reference gene (Rv1506c) or, outside the reference, one sample's copy of a gene (L4_CDC1551_01396), and a column a sample, so the same run shows 1810 genes, 1760 of them in all three, RD4 and RD10 missing from M. bovis only, the phiRv1 prophage from CDC1551 only, and three genes of the RvD2 region that CDC1551 and M. bovis carry and H37Rv lacks. Its files are now PANGENE/pangenome.{gfa,bubbles.txt,Rtab,stat}, one set for the run instead of one per sample, the graph statistics, which were never published, among them",3],[17,"On main / Report","unreleased-report","A report organised by question, results/amap_report.html: next to final_report.html, a new process, AMAP_REPORT, writes one self-contained file from each genome's own outputs: a table of the genomes with the values far from comparable genomes flagged, a map of every reference gene in every genome where regions of difference show as blocks, a lookup of one gene in every genome, the correction from Bakta's calls to AMAP's genes with the genes behind each step, the pseudogenes by lineage, what lies outside the reference, and the functional profile, the categories that differ most between lineages first. It needs only what the correction leaves, so a run with --skip_functional_annotation, which gets no final_report.html, gets this one, and it names the genomes of the sample sheet that produced no annotation. Fifteen MTBC genomes make 0.5 MB (final_report.html is 4.4 MB for twelve); it draws everything with its own JavaScript and needs only Python's standard library to write final_report.html opens with no network: every report carried a script that fetched MathJax from mathjax.rstudio.com, for maths it never shows; mathjax: null leaves it out, so the report opens offline, as the example on the website does. It also declares its language, gives its figures alt text and its tab an icon",3],[17,"On main / Code","unreleased-code","reconcile_coordinates is gone: it read reference coordinates from protein identifiers that have not carried them since March 2026, so it changed nothing. The paralog check does its job with the sample's own offsets",3],[17,"On main / Documentation","unreleased-documentation","A website in plain HTML, drawn with real genes: MkDocs is gone. The front page starts a newcomer in four steps and builds the command for their machine, the pipeline is a metro map whose stations are main.nf's processes, and the four cases, the missing genes and the harder cases (paralogs, small proteins, contig ends, frameshifts, copies) are drawn on genes of CDC1551, M. bovis and E. coli from the benchmark. The tutorial and user guide stay markdown, readable on GitHub, and tools/build_docs.py turns them into pages of the same site, dark by default The site is checked against the code: tools/check_site.py runs every drawn window through bin/, compares the thresholds on the pages with the constants in the code, the metro map with main.nf and every option with the pipeline's parameters, so a change that makes the site wrong fails the docs workflow. tools/figure_data.py redraws the windows from a benchmark run What the docs said that the pipeline does not do: with --skip_functional_annotation a run still writes AMAP/, with the corrected table and sequences, where the outputs page said it did not; no step writes PATHOTYPR/renamed_fastas.tsv; and the 19-residue toxin the algorithm page names is IbsC (b4665), not an ldr toxin A drawing on every page that explains something: the duplication family and communities of CDC1551 and M. bovis, from a real run of both duplication steps; the sample sheet with its tabs and the messages the validator prints; the ranges a genome is validated against; the container and what it holds; a run on a cluster; what happens to a task that fails; the report section by section; the five outcomes and the three inputs of a decision; the command builder on the quickstart. Most are built from the code or the config, so they change when it does The benchmark on the front page: the seven genomes whose answer is known, and how many of their genes AMAP finds, checked against tests/benchmark/README.md. A shared link to any page shows a card drawn from the front page's genes More that the docs got wrong: the duplication page listed ten network metrics, the table has four, and its columns ending in 1 and 2 are the two communities of a pair, not two levels; Nextflow needs Java 17, not 11; --skip_functional_annotation leaves no GFF3 or GenBank, which come after eggNOG; tasks are capped at 16 GB on one machine and at 128 GB under a scheduler profile, not 50 GB; and lineage typing is not skipped on its own for other species, only Mycobrowser is The tutorial's numbers are today's: its case counts and QC line, and their copies on the front page, the outputs page and the custom-reference page, came from runs in July. They now come from the tutorial's own commands run again on 18 October 2026 with --skip_functional_annotation, the QC line from GET_OUTPUTS' own script run on that run's corrected genes: CDC1551 has 4014 genes and 123 pseudogenes as AMAP calls them, where the old line counted Mycobrowser's 13, and Sakai keeps 1387 genes K-12 lacks, not 1356. The tutorial also says where --validate_only writes its verdict and where the number of reference proteins extracted is, as neither reaches the console The README in the site's style: the metro map and the five outcomes in light and dark, the benchmark's scores, and the links a newcomer needs first The README opens with the logo, and its drawing plays: the picture at the top was a still of the front page's drawing, dark whatever GitHub's theme. It now plays the drawing's four steps, in light or dark as the theme is, made by tools/pictures.mjs from the front page's own figure. The README's metro map gains the interactive report's station One DOI: the README, CITATION.cff and every page of the website cited the DOI of one version on Zenodo, and the manifest, main.nf and the interactive report the concept DOI, which always resolves to the latest version. All of them cite the concept DOI now Bakta's options on a SLURM cluster: the HPC page says how to give Bakta a site's options, since for Bakta the QoS the slurm profile sets takes the place of -process.clusterOptions The website is served as it is built: GitHub Pages serves docs/ from main, and by default runs Jekyll over it first, which would have made a page of each markdown file with front matter at the address of the page tools/build_docs.py builds from it. docs/.nojekyll turns Jekyll off, and tools/check_site.py fails without it Website: find anything, read it at any width, and trust what it says: a search on every page, from the header, / or Ctrl+K, finds any section, option, story, front-page question or changelog entry, and each option has a link of its own. Docs pages read in a wider column with their contents on every screen, code that never breaks inside a word, and a missing page that points to the nearest one. Every drawing works by keyboard, touch and screen reader and holds still while stepped through. The front page shows Bakta alone against AMAP, what a working test prints and how long it takes, and commands true for each engine. The pages say what the pipeline does today, in the same words everywhere, and tools/check_site.py checks those claims Website: the questions a first run raises: search answers questions asked in plain words, Troubleshooting says why a sample can be missing from a run that completed, a failed input check is described as it behaves (its report in the terminal, no validation.log), the laptop advice is the same on every page, and links to headings renamed since, the changelog's GitHub anchors among them, still land",3],[17,"On main / Testing","unreleased-testing","tests/selftest/h37rv_selftest.sh annotates H37Rv with itself inside the container and fails unless every GenBank CDS comes back exactly tests/test_cache_keys.py fails when a process runs AMAP code that is not in its cache key, or when its script names a bin/ entry as a bare word without declaring it tests/benchmark/benchmark.sh scores the correction algorithm on seven genomes whose right answer is known (H37Rv twice, CDC1551 complete and in 40 contigs, M. bovis, E. coli K-12 and Sakai), downloading them and running Bakta and miniprot in the container the first time, and lists every gene that changed since an earlier run. A gene whose sequence changed counts as found when a gene with its name covers half of its own bases there, not half of the stretch it lands on, which an insertion can make far longer (63 kb with a prophage inside for b1004 in Sakai); a gene mostly deleted is not counted. Changed genes found: 50 of 50 in CP110619, 860 of 862 in CDC1551, 1515 of 1515 in M. bovis and 3363 of 3367 in Sakai, and the README says why each of the other six is not tests/test_name_by_lineage.py runs PATHOTYPR's own script, rendered as Nextflow renders it, with a stand-in for pathotypr that writes the summaries pathotypr wrote for CDC1551 complete and in 40 contigs, and fails unless Bakta would read a renamed genome as circular exactly when it reads the original so tests/test_sample_ids.py runs TYPE_CASES, GET_OUTPUTS, GET_COUNT, GET_DUPLICATIONS, REPORT_INPUTS and GET_ALLDUP as Nextflow renders them, chained as the pipeline chains them, for a typed and an untyped run whose sample IDs hold underscores, and then the report's own R code where R is installed, and FINAL_REPORT's own script on a run's tables and on one that leaves every table it can empty tests/test_validate_inputs.py runs VALIDATE_INPUTS's own script, rendered as Nextflow renders it, in a task directory laid out as Nextflow lays one out: genomes in a folder the task cannot see, relative paths, two genomes with the same file name and a sheet of 500 samples; and checks both databases, eggNOG's in each state EGGNOG_DB can leave it in tests/test_eggnog_db.py runs EGGNOG_DB's own script, rendered as Nextflow renders it, with its folder staged as Nextflow stages an input and stand-ins for eggNOG-mapper's download and build: the database is built in that folder whatever it is called, one already there is used as it is, and a short bacteria.dmnd is built again tests/test_eggnog_annotation.py runs ANNOT_EGGNOGMAP_BATCH's own script the same way, with a stand-in for emapper.py that looks for the database as eggNOG-mapper does: the database is found whatever its folder is called, a folder called split, where the script writes each sample's annotations, is left as it was, and the tag and log count a single sample as one tests/test_reference_files.py runs the scripts of BAKTA_ANNOTATOR, PATHOTYPR, GET_OUTPUTS and BUNDLED_REFERENCE, rendered as Nextflow renders them, their own Groovy included, with stand-ins for Bakta and pathotypr and each reference file on the host, under a relative path or inside the container; and fails when a process reads a reference file's param other than as what it falls back on when nothing is staged; and checks the Mycobrowser table whenever a step reads it tests/test_pangene_genomes.py runs PANGENE_GENOME and PANGENE's scripts as Nextflow renders them on real genomes of CDC1551, M. bovis and H37Rv cut around RD4 and RvD2 (tests/data/pangenome/, cut again by its make_fixture.py), with stand-ins for miniprot, pangene, k8 and gfatools that hand names on as the real ones do, and fails unless every genome, the sample's own included, is searched with the same proteins, each column is named after its genome, typed or not, a genome none of them is found in keeps a column of zeros, and a protein two samples share goes in once. Where the real tools are installed, as in the container, RD4 must come out missing from M. bovis only and the three RvD2 genes in CDC1551 and M. bovis only, one row each tests/test_report_tables.py merges the matrix PANGENE writes as REPORT_INPUTS does and checks that the report reads every table a # can reach without a comment character; where R is installed, as in the container, it also runs the report's pangenome code on it and counts the genes all its samples share tests/test_cog_counts.py counts COG categories as count_annot.py does, for genes in none, in two and in lowercase, then runs the steps after the correction as tests/test_sample_ids.py chains them, with eggNOG giving some genes no category, then no gene one, then some samples no gene in a category another sample has genes in, and reads what the report's own overview, per-COG figures and statistics table show where R is installed: each category in a figure of its own, with every sample, at 0 where it has no gene. The Mycobrowser figures, statistics and ratios to the median are read the same way for a sample with no gene in one of their categories tests/test_duplication_report.py runs GET_DUPLICATIONS and GET_ALLDUP as Nextflow renders them, chained as the pipeline chains them, with eggNOG giving one copy of a duplicated gene a category, another none and the last no row at all, and checks the columns and COG counts of homologous_communities.tsv; where R is installed, it knits the report's duplication summary on that table and on the one GET_ALLDUP writes when nothing is duplicated. It also fails when a chunk of the report prints markdown without passing it on as markdown tests/test_profile_caps.py fails when a profile sets a cap, or when the order of -profile changes one: it reads nextflow.config and applies its profiles in every order, as both of Nextflow's parsers do, and with Nextflow on PATH also asks nextflow config tests/test_qc_pseudogenes.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling some genes pseudogenes and Mycobrowser marking another, with Mycobrowser on and off, and checks that *-AMAP.qc.tsv, REPORT_INPUTS and the report's own QC table count the pseudogenes AMAP called, as many as the GFF3 and the GenBank mark tests/test_duplication_reproducible.py runs get_duplications.py and network_genomes.py under three hash seeds, on the Diamond tables of three genomes with a family whose average clustering changes with the order of its proteins, and fails unless every file they write is the same under all three; it also fails when either script opens a web browser, with hits or without, as a stand-in browser that records what it is asked to open tells, and when a network page asks a CDN for its drawing library tests/test_custom_reference.py extracts a reference whose plasmid has a gene across its origin from a GenBank record and from GFF3 written either way, on either strand, compares Bakta's calls with it sequence by sequence, and runs PREPARE_REFERENCE's own script, rendered as Nextflow renders it, with a stand-in for Bakta that names sequences as Bakta does tests/test_amap_report.py writes the report from four genomes of a real run cut to two stretches of the reference (tests/data/report/, rebuilt by its make_fixture.py), where M. bovis lacks RD4 and a contig end cuts dnaE2, plus a gene across the origin of a circular plasmid, and checks that every gene, and every Bakta call AMAP did not keep, comes back out of the page as it went in, with Python and, where node is installed, with the page's own JavaScript, along with the states, the flags, the correction's steps and what the process hands the script tests/test_dump_params.py runs DUMP_PARAMS with Nextflow where it is installed, under both parsers, with values holding quotes, $, backticks, a backslash and a line break, and again with -resume and one value changed; everywhere, it fails if the step can come from the cache tests/test_cluster_options.py reads which profile gives Bakta the QoS, and with Nextflow resolves Bakta's and every task's cluster options for each scheduler, in both orders with a container and under both parsers tests/test_environment_pins.py fails when environment.yml pins a tool at a version docker/Dockerfile does not install tests/test_citation.py fails when anything cites AMAP with a DOI other than the concept DOI of the manifest tests/test_clean_proteome.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling some genes pseudogenes, one with a stop codon in its protein and the others without, and Mycobrowser marking a gene AMAP finds intact, with Mycobrowser on and off, and checks that *-AMAP.clean.faa holds every protein FIX_CDS wrote but the pseudogenes', as it wrote them: the CDS the GFF3 does not mark pseudo=true, with the GenBank's translations tests/test_report_pseudogenes.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling a different number of pseudogenes in each sample, none in one, and Mycobrowser marking a gene that every sample has intact, with Mycobrowser on and off, and checks that REPORT_INPUTS hands over each master table row with its sample; where R is installed, it runs the report's own pseudogene section on what REPORT_INPUTS made and checks that it draws each sample at the number AMAP called, the QC table's, a sample with none at 0, under a title that does not name Mycobrowser CI's Python 3.9 job runs on Ubuntu 24.04: GitHub's ubuntu-latest became Ubuntu 26.04, for which setup-python has no Python 3.9, so the job stopped at setup before running a test. 3.9 is the container's Python, so it now runs on 24.04; 3.11 stays on the latest The benchmark scores Bakta alone: tests/benchmark/benchmark.sh scores Bakta's own calls of each genome by the same rules as AMAP's table, in two more columns of its README. Bakta alone finds 67 to 73% of the unchanged genes at exactly their coordinates and under their name (K-12, 90%), where AMAP finds all of them 753 tests, up from 92",3],[17,"1.0.1, 25 July 2026","v1-0-1","Bug fixes, most of them found by running the pipeline on real genomes rather than by reading it. Several changed what the pipeline outputs, so results from 1.0.0 are not directly comparable.",3],[17,"1.0.1 / Correctness","v1-0-1-correctness","Genes no longer vanish during overlap resolution: _insert_with_dedup queued entries for eviction while scanning for conflicts, and applied them even when it then abandoned the insert, so genes were deleted with nothing put in their place Reference pseudogenes are recognised for any organism: the check was gated on the identifier starting with Rv, so on every other organism every reference pseudogene was extended to the next stop and lost its flag. The same gene came out differently depending on what it was called The extracted reference proteome carries its stop codon: extract_ref_proteins.py wrote coordinates spanning the stop but a protein without it, so miniprot's alignment ended three bases short of every Bakta CDS and no gene could ever reach case 2. On E. coli O157:H7 this moved 3325 genes out of conflict resolution Mycobrowser join key: built with rvs.split('_')[0], which returned the whole string for a gene with several references (Rv0094c, Rv3467) and truncated any name containing an underscore (PE_PGRS1 became PE). 44 more genes get a functional category on a real MTBC sample, none lose one Annotation table and FASTA files agree: check_cds.py wrote the FASTA files from dictionaries filled before its final deduplication, and deduplicate_rvs.py matched FASTA headers on the gene id alone, so removing one copy of a multi-copy gene deleted all of them Untranslatable genes are dropped from the table, not left in it with no sequence anywhere",3],[17,"1.0.1 / Crashes","v1-0-1-crashes","--skip_mycobrowser and organism-agnostic mode: the placeholder frame was built with a duplicate Locus column, so pd.merge refused to run and GET_OUTPUTS failed on every run without Mycobrowser miniprot PAF parsing: pandas inferred the column count from the first row, and miniprot varies its trailing tags per alignment, so any file whose first row was narrower aborted the parse Biopython 1.85 and newer: SeqFeature.strand was removed; to_panaroo.py and extract_ref_proteins.py now read it off the location -profile test: pointed at a sample sheet naming paths that are not in this repository. It now annotates the reference genome bundled in the container, so it needs no data and nothing downloaded",3],[17,"1.0.1 / Silent data loss","v1-0-1-silent-data-loss","First Diamond hit: read_and_create left header inference on while stripping # comments, so the first alignment became the column names and disappeared from the duplication analysis Multi-part gene names in rename_fastas.py: headers were cut at the third underscore, collapsing every PE/PPE gene of a sample onto one identifier eggNOG annotation rate: counted with notna(), and emapper writes - for fields it could not fill, so the QC metric read close to 100% whatever the run had done",3],[17,"1.0.1 / Performance","v1-0-1-performance","reconcile_coordinates is linear rather than quadratic: it scanned every protein and every reference for each gene, calling a regex on each candidate. Indexed by gene name up front, a 4000-gene genome goes from 18.3 s to 0.011 s with identical output over 200 randomised genomes deduplicate: list.count() per element and a full scan per duplicated gene, replaced with a Counter and a single grouping pass",3],[17,"1.0.1 / Report","v1-0-1-report","Layout: the stylesheet was losing to the one rmarkdown inlines afterwards, so the article sat underneath the table of contents and the left logo on top of the title Self-contained without a network at render time: jQuery came from a CDN and the second logo from GitHub. pandoc inlines those when the render machine has a network and leaves them dangling when it does not, so a report rendered on an isolated node arrived with a broken image Half the size: 8.2 MB to 4.4 MB, by shipping only the plotly traces actually used and quantising the figures The version in the report is the version that produced it: it was written by hand in three places in the Rmd, so any release that forgot to update them published a report naming the wrong version. It now comes from the manifest in nextflow.config",3],[17,"1.0.1 / Documentation and infrastructure","v1-0-1-documentation-and-infrastructure","Documentation site built with MkDocs Material, published to GitHub Pages, including a tutorial with the numbers from real runs and a rendered example report Docker image workflow: builds, smoke tests and publishes on a v* tag or on demand Tests workflow enabled: it had been sitting disabled and would have failed, since its dependency list was missing plotly and its syntax check had drifted five files behind requirements-test.txt: one dependency list for CI, the docs and a local checkout. environment.yml was missing the Python plotly, so a conda environment built from it could not start classify_cases.py 92 tests, up from 58",3],[17,"1.0.0, 1 April 2026","v1-0-0","",3],[17,"1.0.0 / Correctness","v1-0-0-correctness","Strand-aware gene dictionary: combined_dict keys changed from (Start, End) to (Start, End, Strand): genes on opposite strands at the same coordinates no longer collide silently Consistent 1-based lengths: all length calculations now use End - Start + 1 (15 sites fixed) Robust ID cleaning: regex _\\d+-\\d+$ replaces brittle 4-part underscore rule, so PE_PGRS genes are handled correctly Exact gene name matching: prevents false matches (Rv3508 โ‰  Rv3508c, Rv1 โ‰  Rv10) IntervalTree half-open intervals: all 9 operations corrected from [Start:End] to [Start:End+1] Cross-strand gene preservation: merge operations now include Strand to prevent loss Paralog preservation: multi-copy PE/PPE genes no longer collapsed during groupby Pseudogene propagation: Is_Pseudogene column now reaches master table and report",3],[17,"1.0.0 / Robustness","v1-0-0-robustness","Pre-flight validation: new VALIDATE_INPUTS process checks FASTA format, N-content, GC%, genome size, duplicate headers and database paths, before any computation --validate_only mode: check all inputs without executing the pipeline eggNOG DB integrity: bacteria.dmnd <100 MB detected as corrupted โ†’ automatic rebuild Bakta DB pre-check: actionable error message with download command OOM retry: memory scales with task.attempt (e.g., 50 GB โ†’ 100 GB โ†’ 150 GB) 0-protein handling: eggNOG batch handles empty inputs gracefully Completion handler: prints duration, succeeded/failed/ignored counts",3],[17,"1.0.0 / Reproducibility","v1-0-0-reproducibility","nextflow.manifest: version 1.0.0, Nextflow โ‰ฅ23.04.0, DOI, homepage SOFTWARE_VERSIONS: YAML + TSV with exact versions of 12 tools DUMP_PARAMS: complete parameter dump per run Diamond sorted output: deterministic results regardless of thread count environment.yml: Conda environment with pinned versions Louvain seed: fixed to 42 for reproducible community detection",3],[17,"1.0.0 / Performance and HPC","v1-0-0-performance-and-hpc","Dynamic resource scaling: memory grows with each retry attempt Node-local scratch: --scratch true (default on SLURM/SGE/PBS/LSF) Diamond block size tuning: auto-calculated from available RAM Bakta local temp: --tmp-dir on node-local disk (avoids NFS metadata storms) eggNOG batch mode: single run for all samples (10ร— faster, 50ร— less filesystem I/O) Thread utilization: Bakta 12, Diamond 6-24, miniprot 2ร—3, pangene 2ร—N SLURM/SGE/PBS/LSF profiles: with time limits, queue sizes, rate limiting storeDir for eggNOG DB: persists across runs, survives nextflow clean",3],[17,"1.0.0 / Scalability","v1-0-0-scalability","find -print0 | xargs -0: replaces echo|tr|grep in REPORT_INPUTS (handles 500+ files) publishDir mode: 'link': hardlinks instead of copies for per-sample outputs cache 'lenient': tolerates NFS timestamp drift between cluster nodes awk close(): prevents file handle exhaustion on 500+ sample eggNOG splits JVM tuning docs: NXF_OPTS guidance for large runs",3],[17,"1.0.0 / Organism-agnostic","v1-0-0-organism-agnostic","Custom reference: --ref_annotation + --ref_genome replaces built-in H37Rv Auto-detect mode: --skip_mycobrowser auto-enabled when genus โ‰  Mycobacterium Dynamic reference comparison: report uses median distribution (not hardcoded H37Rv) Auto-extending color palette: non-MTBC lineages get colors automatically Generic labels: \"Ratio to Median\" instead of \"Ratio to H37Rv\" Manifest updated: \"Assisted Microbial Annotator Pipeline\"",3],[17,"1.0.0 / Output quality","v1-0-0-output-quality","Clean proteome: {sample}-AMAP.clean.faa without pseudogenes or internal stops Annotation QC: per-sample metrics (genome size, GC%, coding density, hypothetical %, annotation rate) GFF3 spec compliance: ##sequence-region pragmas, attribute escaping, sorted features QC report section: interactive table + coding density / hypothetical % / genome size plots",3],[17,"1.0.0 / Code quality","v1-0-0-code-quality","58 unit tests: covering IDs, overlaps, reconciliation, validation, clean proteome, QC, merge GitHub Actions CI: Python 3.9 + 3.11 matrix (currently disabled, ready to activate) Type hints: core modules with Coord, Coord3, CoordDict aliases Module docstrings: coordinate conventions, naming schemes documented",3],[17,"1.0.0 / Usability","v1-0-0-usability","Structured startup banner: shows all configuration in a formatted box --version flag: prints pipeline + Nextflow version Progress logging: โ–ธ [BAKTA] Annotating G26689... for every major step Actionable error messages: boxed errors with cause + fix + re-run command Sample count at startup: confirms TSV was parsed correctly",3],[17,"1.0.0 / Documentation","v1-0-0-documentation","README: updated for v1.0.0 with all new parameters, outputs, and organism-agnostic usage CHANGELOG: this file Output tree: documents .clean.faa, .qc.tsv, pipeline_info/ new files",3],[17,"1.0.0 / Bug fixes (85 total)","v1-0-0-bug-fixes-85-total","20 critical fixes and 65 improvements",3],[17,"0.1.0, 2025","v0-1-0","Initial release MTBC-focused annotation pipeline Bakta + miniprot + eggNOG integration HTML report with COG, Mycobrowser, pangenome analysis",3],[18,"","","If AMAP is part of your work, cite it in whichever form your reference manager takes. All three say the same thing; the DOI is the concept DOI, which always resolves to the newest version. APA BibTeX CITATION.cff The CITATION.cff tab is the repository's own file, which GitHub reads to offer a formatted citation from the sidebar.",0,"Ruiz-Rodriguez, P., Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software]. https://doi.org/10.5281/zenodo.8024446 @software{Ruiz-Rodriguez_AMAP_2026, author {Ruiz-Rodriguez, Paula and Mireia}, doi {10.5281/zenodo.8024446}, license {GPL-3.0}, month jul, title {{AMAP}}, url {https://github.com/PathoGenOmics-Lab/AMAP}, version {1.0.1}, year {2026}} cff-version: 1.2.0 title: message: 'If you use this software, please cite it as below.' type: software authors: family-names: Ruiz-Rodriguez given-names: orcid: 'https://orcid.org/0000-0003-0727-5974' affiliation: Institute for Integrative Systems Biology (I2SysBio, University of Valencia CSIC), FISABIO Joint Research Unit โ€œInfection Public Healthโ€, Paterna, Spain email: paula.ruiz.rodriguez@csic.es Coscolla Mireia 'https://orcid.org/0000-0003-0752-0538' mireia.coscolla@csic.es url: 'https://github.com/PathoGenOmics-Lab/AMAP' license: GPL-3.0 version: 1.0.1 date-released: '2026-07-25' doi: 10.5281/zenodo.8024446"],[18,"Authors","authors","Paula Ruiz-Rodriguez and Mireia Coscolla Institute for Integrative Systems Biology (I2SysBio, University of Valencia-CSIC), FISABIO Joint Research Unit \"Infection and Public Health\", Paterna, Spain.",1],[18,"Cite the tools too","cite-the-tools-too","AMAP orchestrates other people's work. If a result depends on one of these, cite it as well. Nextflow: Runs the pipeline, Di Tommaso et al., Nature Biotechnology 35, 316-319 (2017). 10.1038/nbt.3820. Bakta: Baseline annotation, Schwengers et al., Microbial Genomics 7, 000685 (2021). 10.1099/mgen.0.000685. miniprot: Reference proteome alignment, Li, Bioinformatics 39, btad014 (2023). 10.1093/bioinformatics/btad014. eggNOG-mapper: Functional annotation, Cantalapiedra et al., Molecular Biology and Evolution 38, 5825-5829 (2021). 10.1093/molbev/msab293. Diamond: Duplication searches, and eggNOG's, Buchfink et al., Nature Methods 18, 366-368 (2021). 10.1038/s41592-021-01101-x. pangene: Pangenome graph, Li et al., Bioinformatics 40, btae456 (2024). 10.1093/bioinformatics/btae456. pathotypr: Lineage typing of the Mycobacterium tuberculosis complex (MTBC), Its repository, and its lineage markers below. All of them in one block, for a reference manager: The exact version of each one used by a given run is recorded in results/pipeline_info/software_versions.yml.",1,"@article{DiTommaso_Nextflow_2017, author {Di Tommaso, Paolo and Chatzou, Maria Floden, Evan W. Barja, Pablo Prieto Palumbo, Emilio Notredame, Cedric}, title {Nextflow enables reproducible computational workflows}, journal {Nature Biotechnology}, volume {35}, number {4}, pages {316--319}, year {2017}, doi {10.1038/nbt.3820}} @article{Schwengers_Bakta_2021, {Schwengers, Oliver Jelonek, Lukas Dieckmann, Marius Alfred Beyvers, Sebastian Blom, Jochen Goesmann, Alexander}, {Bakta: rapid standardized annotation of bacterial genomes via alignment-free sequence identification}, {Microbial Genomics}, {7}, {11}, {000685}, {2021}, {10.1099/mgen.0.000685}} @article{Li_miniprot_2023, {Li, Heng}, {Protein-to-genome alignment with miniprot}, {Bioinformatics}, {39}, {1}, {btad014}, {2023}, {10.1093/bioinformatics/btad014}} @article{Cantalapiedra_eggNOGmapper_2021, {Cantalapiedra, Carlos P. Hern{\\'a}ndez-Plaza, Ana Letunic, Ivica Bork, Peer Huerta-Cepas, Jaime}, {{eggNOG-mapper} v2: Functional Annotation, Orthology Assignments, Domain Prediction at the Metagenomic Scale}, {Molecular Biology Evolution}, {38}, {12}, {5825--5829}, {10.1093/molbev/msab293}} @article{Buchfink_DIAMOND_2021, {Buchfink, Benjamin Reuter, Klaus Drost, Hajk-Georg}, {Sensitive protein alignments tree-of-life scale using {DIAMOND}}, Methods}, {18}, {366--368}, {10.1038/s41592-021-01101-x}} @article{Li_pangene_2024, Heng Marin, Maximillian Farhat, Maha R.}, {Exploring gene content pangene graphs}, {40}, {btae456}, {2024}, {10.1093/bioinformatics/btae456}}"],[18,"Reference data","reference-data","The MTBC reference files bundled in the container come from public repositories. The pathotypr lineage markers are archived on Zenodo under 10.5281/zenodo.19210044, and Mycobrowser functional categories come from mycobrowser.epfl.ch.",1],[18,"Licence","licence","AMAP is released under the GPL-3.0 licence. See LICENSE.",1],[19,"","","Bug reports, fixes and suggestions are all welcome. The full guidelines live in CONTRIBUTING.md, and the project follows a code of conduct.",0],[19,"Reporting a bug","reporting-a-bug","Open an issue at github.com/PathoGenOmics-Lab/AMAP/issues with the command you ran, the failing task's .command.err, and both software_versions.yml and params.yml from pipeline_info/. Those last two pin the exact versions and parameters, which is usually what the answer turns on. Troubleshooting covers the common cases first.",1],[19,"Working on the code","working-on-the-code","requirements-test.txt covers everything bin/ imports, not only what the tests import directly, and CI installs from the same file. A shorter list appears to work until a test touches classify_cases.py, which pulls in plotly through classify/plotting.py. The suite covers ID handling, overlap arithmetic, case classification, the rules that resolve and recover genes, and the parsers, and it runs in about three minutes (some tests skip themselves when a tool such as R or Diamond is missing). Anything touching bin/classify/ should come with a test, since the correction logic is where a silent change does the most damage.",1,"git clone https://github.com/PathoGenOmics-Lab/AMAP cd AMAP pip install -r requirements-test.txt python -m pytest tests/ -v"],[19,"Annotating H37Rv with itself","annotating-h37rv-with-itself","A change to the correction algorithm should also pass the self-test, which has exactly one right answer: annotating H37Rv against its own reference must give back every CDS of the AL123456.3 GenBank record, at exactly its coordinates and strand, under its own name. The first run spends a couple of minutes on Bakta and miniprot and keeps their output under selftest_h37rv/. After that it only runs classify_cases.py, check_cds.py and deduplicate_rvs.py, in seconds, and exits non-zero with the offending genes listed if anything is missing, misplaced or misnamed. H37Rv is one genome, and the one the defaults were tuned on. The accuracy benchmark in tests/benchmark/ scores six more whose right answer is known: another H37Rv assembly, CDC1551 complete and cut into 40 contigs, M. bovis, and E. coli K-12 and Sakai against K-12 as a custom reference. Give a run a name, and the next one lists every gene that changed: tests/benchmark/README.md explains how the truth is built and what to expect. The quickest check of the whole chain up to the correction is the test profile, without eggNOG, whose 45 GB database the test profile's 8 GB cap cannot hold:",1,"docker run --rm --platform linux/amd64 -v \"$PWD:/amap\" -w /amap paururo/amap:latest bash tests/selftest/h37rv_selftest.sh GenBank CDS: 4031 AMAP genes: exact and named right: not at exactly their coordinates: 0 wrong: genes that are no tests/benchmark/benchmark.sh benchmark_work before change the algorithm, then after nextflow -profile docker,test --skip_functional_annotation --outdir amap-test"],[19,"Pipeline code and -resume","pipeline-code-and--resume","Nextflow hashes a task's inputs and script, not the files the script calls by path, so python3 ${projectDir}/bin/check_cds.py alone would let -resume hand back what the old check_cds.py produced. Every process that runs AMAP code therefore lists those files in ext code, and its script names task.ext.code in a comment, which puts them in the cache key (Nextflow 24.04 and later): A script that imports classify needs classify/*.py in the list. The reverse also bites: Nextflow adds to a task's cache key any bin/ entry its script names as a bare word, which is why PATHOTYPR quotes its 'classify' subcommand. tests/test_cache_keys.py fails when a process runs a file it does not list, lists one that does not exist, or names a bin/ entry it does not declare.",1,"ext code: files(\"${projectDir}/bin/{check_cds.py,deduplicate_rvs.py,classify/*.py}\")"],[19,"Working on the container","working-on-the-container","The smoke test checks the tools, the Python and R libraries, the bundled reference data, and runs the test suite with the image interpreter. Worth doing before proposing a change to the Dockerfile, since a library update can break the pipeline scripts without the image looking any different.",1,"docker build --platform linux/amd64 -f docker/Dockerfile -t amap:dev docker/smoke_test.sh"],[19,"Working on these docs","working-on-these-docs","The website is plain HTML in docs/, with nothing to build on the server: GitHub Pages serves the folder as it is (docs/.nojekyll keeps it from running Jekyll over the markdown), and it opens from a clone too. The long pages are the markdown files, which GitHub also renders, and tools/build_docs.py writes each one's .html next to it, with the site's header, page list, contents and search. The front page (docs/index.html) and the illustrated walk through the correction (docs/how-it-works.html) are written by hand. Their drawings come from docs/assets/figures.js and how.js, and every gene in them from the windows of real runs in docs/assets/figure-data.js. A line in a markdown page puts one of those drawings there, and GitHub shows nothing in its place. Some drawings are made from the code itself when the page is built, so they cannot drift from it: the sample sheet and its error messages run bin/validate_inputs.py, the validation ranges read its limits, the container reads docker/Dockerfile, and the cluster and failure drawings read nextflow.config. After changing any of those, run build_docs.py again. check_site.py fails when a page no longer matches its markdown, a link or an anchor is broken, a page uses an option the pipeline does not have, a threshold on a page differs from the code, the metro map's stations are out of the order main.nf runs them in, or a drawing shows something bin/ would not do: it runs every window of figure-data.js through the classification, check_cds.py and deduplicate_rvs.py. When a change to the algorithm changes a drawn window, rerun the benchmark, write the windows again, and read the words of the drawing against what changed: The duplications page draws a real family and a pair of communities from docs/assets/dup-data.js, which check_site.py puts back through get_duplications.py and network_genomes.py. tools/dup_run.sh runs the two duplication steps in the container on corrected proteomes, and tools/dup_data.py writes the data from what they found (both explain their arguments). The front page's benchmark scores are checked against tests/benchmark/README.md. The pictures, the example report's sections, the card a shared link shows and the README's light and dark drawings are screenshots of the site taken by headless Chrome; take them again after the report or a drawing changes: The workflow in .github/workflows/docs.yml runs both checks on every pull request that touches the docs, bin/, modules/, main.nf, nextflow.config, docker/Dockerfile or the benchmark's expected results.",1,"python3 tools/build_docs.py after editing any .md under docs/ tools/check_site.py what CI runs -m http.server -d docs 8000 then open http://localhost:8000 tools/figure_data.py benchmark_work final --reference /reference node tools/pictures.mjs"],[19,"Style","style","Keep to what is already there: short paragraphs, GitHub's alerts (> [!WARNING]) for the things that actually bite, and concrete numbers rather than \"large\" or \"slow\". Where a threshold appears in the docs it should match the constant in the code, so if you change one, change the other; check_site.py holds the ones it knows to the code.",1],[20,"","","AMAP, the Assisted Microbial Annotator Pipeline, annotates assembled bacterial genomes with Bakta, aligns a trusted reference proteome onto them with miniprot, and referees every disagreement. Built for the Mycobacterium tuberculosis complex, usable with any reference.",0],[20,"From nothing to a corrected genome","start","From nothing to a corrected genome Every tool AMAP calls lives in one container image, so Nextflow and a container engine are all you install. Four steps; once the image is in, the slowest takes minutes.",1],[20,"Each call, held up against the reference","cases","Each call, held up against the reference Every Bakta call is compared with where the reference proteins align, on its strand and contig. That puts it in one of four cases, numbered as the code numbers them: 2 and 3 agree, 4 conflicts, 1 has no reference. A reference gene with no call at all is a fifth. These are real genes of CDC1551: point at tap one for its coordinates.",1],[20,"Checked against genomes whose answer is known","proof","Checked against genomes whose answer is known Seven genomes, scored against genes AMAP did not annotate: the reference's own, carried onto each genome by a whole-genome alignment. AMAP finds every unchanged gene at exactly its coordinates, under its name; Bakta alone, scored the same way, finds 67 to 90% of them. 100% of unchanged genes found at exactly their coordinates, in all seven genomes. Bakta alone: 67 to 90%.",1],[20,"Where each step happens","pipeline","Where each step happens Stations are the processes of main.nf; lines are what travels between them. Pick a station, or flip an option to see what a run leaves out.",1],[20,"Files you can open, and the reasons behind them","outputs","Files you can open, and the reasons behind them Everything lands under results/, one folder per step. Typing puts the lineage in front of each sample, so CDC1551 becomes L4_CDC1551; --skip_typing leaves it out. Outputs lists every file.",1],[20,"The whole run on one page","report","The whole run on one page This one is the example report: twelve made-up genomes across four lineages, built around a real CDC1551 run. Synthetic data, so read the layout, not the numbers.",1],[20,"The questions people ask first","faq","The questions people ask first",1],[20,"How long does a run take?","faq","Minutes for a genome, without eggNOG. Measured on a MacBook Pro with an M4 Pro, 14 cores, Docker Desktop given 8 GB, the image under emulation and already pulled: The check of step 3, one genome 9 s The test of step 2 2 min 47 s M. tuberculosis CDC1551 without eggNOG, the tutorial's run 2 min 16 s E. coli Sakai against K-12, your own reference, without eggNOG 5 min 47 s Bakta takes most of a run: 1 min 46 s of the CDC1551 one. With your own reference it runs on the reference too, 2 min 36 s for K-12. The eggNOG steps are not timed here: they ask for 45 and 50 GB of memory, more than a laptop gives Docker. The database is built once, on the first full run, and kept where --eggnog_db points. What each step needs.",4],[20,"The sample sheet is rejected","faq","It has to be separated by a real tab: spaces will not do, and this is the most common reason. Check with cat -t samples.tsv, where real tabs show up as ^I. Keep sample IDs to letters, digits and underscores.",4],[20,"Nextflow will not start: \"Cannot find Java or it's a wrong version\"","faq","Nextflow needs Java 17 or newer (up to 26), and java -version shows the one it found. Install one with conda install -c conda-forge openjdk=17, or point JAVA_HOME at a newer JDK.",4],[20,"A task dies with exit code 137","faq","On macOS and Windows, Docker runs inside a virtual machine with a memory limit of its own, often 8 GB; docker info, from step 1, prints it. Raise it in Docker Desktop under Settings, Resources, Memory, and keep --max_memory at or below it. What each step needs.",4],[20,"The first run seems stuck","faq","The first run pulls the image, 3.4 GB compressed and 12 GB on disk, and builds the eggNOG database, which dominates the wall clock; both happen once. A long silence during the database build is not a failure. Point --eggnog_db somewhere you keep, and -resume picks a stopped run up where it left off.",4],[20,"Case 2 is at or near zero","faq","Every gene disagreeing with the reference by a few bases means something systematic is wrong with the reference, not with 4,000 genes. The usual cause is a reference annotation from another assembly than the genome given with it. What a healthy summary looks like.",4],[20,"Does it work for anything but tuberculosis?","faq","Yes. Give it a trusted annotation and the genome it describes with --ref_annotation and --ref_genome, and turn off the two steps that are for the tuberculosis complex only. Using your own reference.",4],[20,"A task failed. Where do I look?","faq","Nextflow prints the work directory of the failed task. Its .command.err is almost always the answer, and .command.sh is the exact script that ran. If the run finished but a sample is missing, its task was set aside after two more tries: see a sample is missing. Troubleshooting, by error.",4],[21,"","","One gene at a time, from Bakta's call to what AMAP keeps.",0],[21,"Agrees","agree","Bakta calls dnaN from 2,052 to 3,260 on the plus strand. miniprot aligns H37Rv's Rv0002, dnaN, onto CDC1551: the whole protein, from 2,052 to 3,260, stop codon included. Same start, same end, same strand and the same length: case 2, a perfect match. Nothing to settle. The call stays as Bakta made it, and is tied to Rv0002. It ends in a stop codon: nothing to change. One gene for each reference protein here: nothing to sort out. It keeps Bakta's locus tag and name, with Rv0002 in its Dbxref. Case 2. The five outcomes, in eight stories.",5],[21,"Same ends, other length","same-ends","Bakta calls fhaA from 23,861 to 25,426 on the minus strand: 1,566 bp. H37Rv's Rv0020c, fhaA, aligns over exactly the same stretch, the whole protein to its stop codon. Its protein is six residues longer: miniprot aligns 1,584 nt of it on these 1,566 bp, with a gap where CDC1551 lacks them. Same start, same end, same strand, but not the same length: 1,566 bp called, 1,584 nt aligned. That is case 3, an imperfect match. Nothing to settle. A case 3 call is reliable enough: it stays as Bakta made it, tied to Rv0020c. It ends in a stop codon: nothing to change. One gene for each reference protein here: nothing to sort out. It keeps Bakta's locus tag, name and product, with Rv0020c in its Dbxref. Case 3. The five outcomes, in eight stories.",5],[21,"Starts late","late-start","Bakta calls dnaA from base 34 to 1,524. H37Rv's Rv0001, dnaA, aligns end to end from base 1 to 1,524. Same strand, same end, but a start 33 bp apart: case 4, a conflicting overlap. One call against one alignment. With a single overlap the reference wins, unless the call is longer by more than 550 bp and the alignment is not complete in its frame. Here the alignment is the longer of the two: AMAP keeps 1 to 1,524. It ends in a stop codon: nothing to change. One gene for each reference protein here: nothing to sort out. No Bakta call has these coordinates, so it is written under the reference protein's name. Case 4. The five outcomes, in eight stories.",5],[21,"One gene, two calls","split","Bakta calls two short genes, from 9,963 to 10,160 and from 10,523 to 10,828, both hypothetical proteins. H37Rv's Rv0007 aligns end to end over both, from 9,914 to 10,828. Both calls overlap the one alignment with other coordinates: case 4, with several calls on one alignment. The call that overlaps the alignment most is weighed against it. The reference wins if it is at least as long, if the call is longer by less than 500 bp, if the alignment is complete in the call's frame, or if the call spans other reference genes too. Rv0007 is longer than either call: one gene goes in, 9,914 to 10,828. It ends in a stop codon: nothing to change. One gene for each reference protein here: nothing to sort out. Written under Rv0007, as no Bakta call has its coordinates. Case 4. The five outcomes, in eight stories.",5],[21,"Much longer","longer","Bakta calls a 4,821 bp site-specific DNA methyltransferase on the minus strand. H37Rv's Rv2024c, 516 residues, aligns over 1,518 bp at one end of it: residues 1 to 506, short of the protein's end, with no stop codon. One call against one alignment, with other coordinates: case 4. A call longer by more than 550 bp wins over an alignment that is not complete in its frame: such an alignment stopped short, and dropping a whole gene for it would be the worse mistake. Bakta's call, 3,303 bp longer, is kept. It ends in a stop codon: nothing to change. One gene for each reference protein here: nothing to sort out. It keeps Bakta's locus tag and product, and takes Rv2024c's name in its Dbxref: no other gene carries it, and the alignment spans more than half of the protein. Case 4. The five outcomes, in eight stories.",5],[21,"Not in the reference","new","Bakta calls four genes in a row: a mannosyltransferase, a sulfite oxidase-like oxidoreductase, an MMPL transporter and a hypothetical protein. No H37Rv protein aligns anywhere along them. A call with no alignment over it on its strand is case 1, potentially novel. A case 1 call goes if Bakta's known false positives cover 80% of it, or if its protein matches the H37Rv genome at 80% identity or more. None is covered, three of the proteins do not align to H37Rv at all and the fourth matches at 60%: all four are kept. They end in stop codons: nothing to change. No reference protein to share: nothing to sort out. They keep Bakta's locus tags and products, with no reference name. Case 1. The five outcomes, in eight stories.",5],[21,"A known false call","false","Bakta calls a 108 bp hypothetical protein on the minus strand. No H37Rv protein aligns here. Bakta's known false positives are aligned too: calls Bakta makes on H37Rv itself that its curated annotation does not have. One covers this call end to end. Case 1, potentially novel. Covered 100% by a known false positive, more than the 80% that rules a call out: it is dropped. Nothing left here to translate. Nothing to sort out. Nothing is written here. Case 1. The five outcomes, in eight stories.",5],[21,"Missed by Bakta","missed","Bakta calls a gene from 33,168 to 33,497 and bioF2 from 34,239, and nothing between them. H37Rv's Rv0031, 71 residues, aligns between the two, end to end, from 33,526 to 33,738. An alignment with no call over it on its strand: a missing reference gene. The two calls match Rv0030 and Rv0032 exactly, case 2. A protein is missing when no gene accounts for any of its placements: none sits on it, in the same frame, covering half of it. So Rv0031 goes back in at its best placement. It ends in a stop codon: nothing to change. One gene for each reference protein here: nothing to sort out. Written under Rv0031. Missing. The five outcomes, in eight stories.",5],[21,"On its paralog's locus","paralog","Two loci of CDC1551, about 208 kb apart. Bakta calls a REP13E12 repeat protein at each, from 400,321 and from 607,987. H37Rv's Rv0336 and Rv0515 are paralogs, and miniprot places each at both loci, as it reports every placement within 5% of a protein's best. The proteins placed only once around each locus, five on each side, say where it sits in H37Rv: the first at about 400 kb, where Rv0336 belongs, the second at about 607 kb, where Rv0515 does. Each protein's placement on the other's locus is dropped. Each REP13E12 call now overlaps one alignment, starting 69 bp after it: case 4 at both loci. The calls around them match their proteins exactly, case 2. The reference wins both: Rv0336 at 400,252 and Rv0515 at 607,918. Both end in stop codons: nothing to change. One gene for each reference protein here: nothing to sort out. Each locus is written under its own protein. Paralogs. The harder ones.",5],[21,"Too small for miniprot","small","E. coli K-12 against its own annotation. Bakta calls fau and serA, and nothing in the 388 bp between them. miniprot places fau and serA. IbsC, a toxic peptide of 19 residues, is too small to seed an alignment of its own: miniprot placed it only at 3,194,766, on its neighbour IbsD's locus, where 3 of its residues differ. Every protein of 8 residues or more without an exact, complete placement is looked for in the six-frame translation of the genome, a start codon, its residues, then a stop: IbsC is here, from 3,056,890 to 3,056,949 on the minus strand. fau: Bakta's call starts 54 bp before the alignment, case 4. serA matches exactly, case 2. IbsC has no call over it: a missing reference gene. fau takes the reference's start, and IbsC goes in where the exact search found it. All three end in stop codons: nothing to change. One gene for each reference protein here: nothing to sort out. IbsC is written under its own name, b4665. Exact search. The harder ones.",5],[21,"Cut by a contig end","contig-end","CDC1551 cut into 40 contigs. Near the end of contig_10 Bakta calls Rv1240's gene and two short genes on the minus strand, and nothing on the plus strand. At the end of contig_9 it calls sugC and a short hypothetical protein. H37Rv's Rv1239c, 367 residues, aligns on the plus strand from 101,478 to the very end of contig_10, 102,146: its first 223 residues. Its last 144, which miniprot does not report beside the larger piece, are aligned again on their own, and land at the very end of contig_9: the other piece. Rv1240's call and sugC's match exactly, case 2. The two short calls on contig_10 have nothing aligned, case 1. The short call on contig_9 overlaps the piece with other coordinates, case 4. On contig_10 Rv1239c has no call over it on its strand: a missing reference gene. Rv1239c goes back in at its placement on contig_10, and on contig_9 the piece wins over the shorter call, as the reference does. The two short calls on contig_10 go: their proteins match the H37Rv genome at 98 and 99% identity, so they are not new. Each piece's frame reaches its contig's end with no stop codon, from within 30 bp of it: the assembly cut the gene. The piece on contig_10 is marked partial at its 3' end, the one on contig_9 at its 5' end. Neither is a pseudogene. Two genes carry Rv1239c, but they are the pieces of one cut gene, not copies, and both stay. Their IDs must differ: one is written Rv1239c and the other Rv1239c_2, both with Rv1239c in their Dbxref. The cut ends are written the way NCBI writes them: partial=true with end_range in the GFF3, and a > before the cut end in the GenBank location. Partial. The harder ones.",5],[21,"Runs into the next gene","frameshift","M. bovis AF2122/97. Bakta calls one gene from 689,275 to 690,306, between mce2A's and mce2C's. Two H37Rv proteins align inside it: Rv0590 end to end, with one base too many for its frame, and Rv0590A after it. The call overlaps both alignments with other coordinates: case 4. A call over more than one alignment is several reference genes read as one, and the reference wins: Rv0590 and Rv0590A go in as two genes. Rv0590 has no stop codon where its protein ends. The next stop in its frame is 67 residues past the end of the protein, at 690,306, the end of Rv0590A and of Bakta's call. An extension may reach 10 residues past the protein. So Rv0590 keeps its extent and is flagged a pseudogene, with the reason. One gene for each reference protein here: nothing to sort out. The pseudogene is written the way NCBI writes one: pseudo=true and the reason in Note, /pseudo and /note in the GenBank, with no translation. Pseudogene. The harder ones.",5],[21,"Two real copies","copies","E. coli O157:H7 Sakai, against K-12. Bakta calls a 4.2 kb RhsD gene at 617,058 and another at 2,046,050, 1.4 Mb apart. K-12's RhsD (b0497) aligns at both, over most of each gene but short of its end. Each RhsD call overlaps its alignment with other coordinates, case 4. The other calls have nothing aligned, case 1: the three at the second locus are kept, and the short one at the first goes, as one of the calls Bakta makes on K-12 itself covers it. Each call is longer than its alignment by less than 550 bp: the reference wins at both. Neither alignment reaches a stop codon. Each gene is extended to the next stop in its frame, which completes the protein and is where Bakta's calls end. Two genes now carry b0497. A genome can have a gene twice: the second stays, as it is at least 90% as long as the first, clear of it and on no other reference gene's locus. The copy later in the genome is named b0497_2, and the duplication analysis counts it. Both copies are written, each with an ID of its own. Copies. The harder ones.",5]]}); diff --git a/docs/assets/site.css b/docs/assets/site.css index 79b45aa..1a9836b 100644 --- a/docs/assets/site.css +++ b/docs/assets/site.css @@ -3,38 +3,148 @@ are the whole visual language: what Bakta called, where the reference protein aligned, and what AMAP keeps. */ -/* Dark by default, whatever the system says; light only when chosen with the - toggle (its key is amap.theme). */ +/* โ”€โ”€ Design tokens โ”€โ”€ + One system for every page. Type: a modular scale, 17px x 1.125^n (a major + second), named by its step. Space: a 4px grid. Radii: five steps, the + inner of two nested shapes one step under the outer. Surfaces: the page, + then three levels up and one down (code). Ink: four levels. The three + track colours are data: they mean Bakta, the reference and AMAP, and are + not used to decorate. Dark ("Noche") by default, whatever the system says; + light only when chosen with the toggle (its key is amap.theme). */ :root { - --bg: #0b0f0e; - --bg-elev: #121716; - --bg-subtle: #19201e; - --ink: #eef4f2; - --ink-soft: #cdd7d4; - --muted: #98a6a2; - --line: #222b29; - --line-strong: #34413e; + /* Type scale, 17px x 1.125^n. */ + --fs-n3: 0.7465rem; /* 11.9px caps labels: kickers, table heads, tags */ + --fs-n2: 0.8398rem; /* 13.4px captions, legends, meta, code blocks */ + --fs-n1: 0.9448rem; /* 15.1px interface: nav, buttons, cards, tables */ + --fs-0: 1.0625rem; /* 17px body */ + --fs-1: 1.1953rem; /* 19.1px lead, card titles, h4 */ + --fs-2: 1.3447rem; /* 21.5px h3 */ + --fs-3: 1.5128rem; /* 24.2px stepper title */ + --fs-4: 1.7019rem; /* 27.2px docs h2 */ + --fs-6: 2.1540rem; /* 34.5px h2 and h1 on a phone */ + --fs-7: 2.4233rem; /* 38.8px section h2, docs h1 */ + --fs-10: 3.4504rem; /* 55.2px */ + --fs-11: 3.8817rem; /* 62.1px the front page's h1 */ + /* Display sizes move with the screen between two steps: a title from 27px + on a phone to 38.8px from 1000px; the front page's h1 from 34.5px to + 62.1px. */ + --fs-title: clamp(var(--fs-4), 1.32rem + 1.77vw, var(--fs-7)); + --fs-hero: clamp(var(--fs-6), 5.2vw, var(--fs-11)); + --lh-display: 1.1; + --lh-heading: 1.2; + --lh-label: 1.3; + --lh-ui: 1.45; + --lh-read: 1.55; + --lh-body: 1.6; + --fw-light: 300; + --fw-regular: 400; + --fw-medium: 500; + --fw-semibold: 600; + --fw-bold: 700; + --fw-display: 760; + --tr-hero: -0.038em; + --tr-display: -0.032em; + --tr-heading: -0.016em; + --tr-numeral: -0.035em; + --tr-caps: 0.1em; + /* Motion: a control's change of state; a change a reader should notice + (the map's lines); a layer of a drawing coming in. */ + --dur-state: 0.15s; + --dur-emph: 0.3s; + --dur-reveal: 0.45s; + --ease: ease; + /* Space, a 4px grid. */ + --sp-1: 4px; + --sp-2: 8px; + --sp-3: 12px; + --sp-4: 16px; + --sp-5: 20px; + --sp-6: 24px; + --sp-7: 32px; + --sp-8: 40px; + --sp-9: 48px; + --sp-10: 64px; + --sp-11: 80px; + --sp-12: 96px; + /* Radii: marks, small controls, controls, cards, plates. */ + --r-1: 4px; + --r-2: 6px; + --r-3: 8px; + --r-4: 12px; + --r-5: 16px; + --r-pill: 999px; + /* Layout: the front page's column, and the margin column beside its main + one, where the numbers of chapters and steps hang. The margin column and + its gap make whole squares of the graticule (three, four from 1216px), + so the main column starts on a major line, where the docs' text does. */ + --wrap: 1216px; + --grid-sq: 80px; + --col-gap: var(--sp-9); + --col-margin: calc(3 * var(--grid-sq) - var(--col-gap)); + /* How far in from its edge a plate sets what it holds. */ + --plate-pad: var(--sp-6); + /* The sticky header, its border included; the bar a phone adds under it; + and where a page's own sticky boxes and jumped-to headings come to rest + below it (where the docs' sidebar sticks). */ + --header-h: 61px; + --here-h: 40px; + --clear: calc(var(--header-h) + var(--sp-5) - 1px); + --font: -apple-system, BlinkMacSystemFont, "Segoe UI", Inter, Roboto, "Helvetica Neue", Arial, sans-serif; + --mono: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; + + /* Surfaces: sunk (code) < page < card < plate < fill. */ + --surface-sunk: #070a09; + --surface-0: #0a0e0d; + --surface-1: #101615; + --surface-2: #141b1a; + --surface-3: #1b2422; + /* Ink: headings, body, captions, decoration (never text). */ + --ink-1: #eef4f2; + --ink-2: #c3cfcb; + --ink-3: #93a29e; + --ink-4: #6b7975; + /* Lines: hairline, edge, and the 3:1 edge of a field you type in. */ + --line-1: #1c2523; + --line-2: #29332f; + --line-3: #5c6b67; + /* The one accent, and its wash (12% over a card, opaque). */ + --accent: #4fd6bf; + --accent-hover: #86e8d7; + --accent-wash: #182d29; + --accent-line: #2c6f63; + --on-accent: #04201a; + /* Brand: the logo's own colours, for the headline and the logo only. */ --brand-teal: #39baa3; --brand-aqua: #68e9d4; --brand-slate: #8faeb7; --grad-a: #7ff0dc; --grad-b: #39baa3; --grad-c: #8faeb7; - --accent: #4fd6bf; - --accent-hover: #86e8d7; - --accent-soft: #10292a; - --on-accent: #062019; - --code-bg: #080b0a; - --code-border: #1f2826; + /* Elevation: 1 a card (edge, no shadow), 2 a plate, 3 an overlay. */ + --shadow-1: none; + --shadow-2: inset 0 1px 0 rgba(255, 255, 255, 0.035), 0 1px 2px rgba(0, 0, 0, 0.5), 0 18px 40px -18px rgba(0, 0, 0, 0.7); + --shadow-3: 0 1px 2px rgba(0, 0, 0, 0.5), 0 24px 56px -12px rgba(0, 0, 0, 0.6); + --glow: 0 0 0 1px rgba(79, 214, 191, 0.5), 0 6px 20px -6px rgba(79, 214, 191, 0.55); + /* Context: what a control sits on decides its fill (one level up), and a + drawing's halos take the colour of the surface under them. */ + --control: var(--surface-1); + --control-raised: var(--surface-3); + --halo: var(--surface-1); + --overlay: var(--surface-3); + + /* Code. */ + --code-bg: var(--surface-sunk); + --code-border: var(--line-1); --code-ink: #e3ebe8; --code-muted: #889692; --code-flag: #7ee2cf; --code-value: #f2c47c; - --copy-border: rgba(255, 255, 255, 0.16); - --copy-bg: rgba(255, 255, 255, 0.06); - --copy-hover: rgba(255, 255, 255, 0.14); + --copy-border: rgba(255, 255, 255, 0.14); + --copy-bg: rgba(255, 255, 255, 0.05); + --copy-hover: rgba(255, 255, 255, 0.12); --copy-done: #8ef0bf; - /* The tracks: Bakta's calls, the reference protein aligned, what AMAP keeps. */ + /* Data: the tracks (Bakta's calls, the reference protein aligned, what AMAP + keeps) and the states a drawing marks. */ --bakta: #f3a84a; --ref: #a99cff; --amap: #3fd6bb; @@ -52,49 +162,78 @@ --m-db: #f3a84a; --m-dup: #6fb6ff; --m-pan: #e889d8; - --radius: 12px; - --radius-sm: 8px; - --shadow: 0 1px 2px rgba(0, 0, 0, 0.4), 0 8px 24px rgba(0, 0, 0, 0.35); - --font: -apple-system, BlinkMacSystemFont, "Segoe UI", Inter, Roboto, "Helvetica Neue", Arial, sans-serif; - --mono: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; + /* The graticule the pages sit on: minor lines every 16px, major every 80 + (five squares), in the accent at a breath of alpha; a ruler's ticks in + the logo's slate; the corner marks of a plate in the accent. */ + --grid-minor: rgba(104, 233, 212, 0.028); + --grid-major: rgba(104, 233, 212, 0.06); + --tick: rgba(143, 174, 183, 0.34); + --tick-major: rgba(143, 174, 183, 0.62); + --corner: color-mix(in srgb, var(--accent) 70%, transparent); + + /* The names the rest of this sheet uses, kept as aliases of the system. */ + --bg: var(--surface-0); + --bg-elev: var(--surface-1); + --bg-subtle: var(--surface-3); + --ink: var(--ink-1); + --ink-soft: var(--ink-2); + --muted: var(--ink-3); + --line: var(--line-1); + --line-strong: var(--line-2); + --line-input: var(--line-3); + --accent-soft: var(--accent-wash); + --radius: var(--r-4); + --radius-sm: var(--r-3); + --shadow: var(--shadow-2); color-scheme: dark; } :root[data-theme="light"] { - --bg: #fbfcfb; - --bg-elev: #ffffff; - --bg-subtle: #f1f5f4; - --ink: #121a18; - --ink-soft: #36423f; - --muted: #5a6763; - --line: #e0e7e5; - --line-strong: #c8d2cf; - --grad-a: #0b8a75; - --grad-b: #2f9e8a; - --grad-c: #4f7682; - --accent: #087a67; - --accent-hover: #055c4d; - --accent-soft: #e2f3ef; + --surface-sunk: #eef2f1; + --surface-0: #f5f7f6; + --surface-1: #ffffff; + --surface-2: #ffffff; + --surface-3: #ebf0ee; + --ink-1: #0f1715; + --ink-2: #2e3a37; + --ink-3: #53605c; + --ink-4: #7a8783; + --line-1: #e2e8e6; + --line-2: #d0d9d6; + --line-3: #7c8985; + --accent: #076d5c; + --accent-hover: #05584a; + --accent-wash: #e6f1ef; + --accent-line: #8cc5ba; --on-accent: #ffffff; - --code-bg: #f3f6f5; - --code-border: #dfe6e4; + --brand-teal: #155b4e; + --brand-aqua: #11978b; + --brand-slate: #4f7682; + --grad-a: #155b4e; + --grad-b: #0b8a75; + --grad-c: #4f7682; + --shadow-2: 0 1px 2px rgba(16, 27, 24, 0.06), 0 16px 36px -18px rgba(16, 27, 24, 0.22); + --shadow-3: 0 1px 3px rgba(16, 27, 24, 0.08), 0 24px 56px -16px rgba(16, 27, 24, 0.24); + --glow: 0 1px 2px rgba(7, 109, 92, 0.25), 0 6px 18px -8px rgba(7, 109, 92, 0.6); + --control-raised: #ffffff; + --overlay: #ffffff; --code-ink: #1a2422; - --code-muted: #67736f; - --code-flag: #087a67; - --code-value: #9a4f00; - --copy-border: rgba(16, 27, 24, 0.14); - --copy-bg: rgba(255, 255, 255, 0.8); - --copy-hover: #ffffff; + --code-muted: #616e6a; + --code-flag: #076d5c; + --code-value: #8f4a00; + --copy-border: rgba(16, 27, 24, 0.16); + --copy-bg: #ffffff; + --copy-hover: color-mix(in srgb, var(--ink-1) 6%, #ffffff); --copy-done: #0f7a52; --bakta: #a85a06; --ref: #5845cc; - --amap: #08806b; - --drop: #98a3a0; + --amap: #07796a; + --drop: #7d8986; --on-track: #ffffff; --pseudo: #c0244a; - --ok: #0f7a52; + --ok: #0c724d; --note: #b7791f; - --warn: #a8600a; + --warn: #9b5806; --danger: #c0244a; --info: #1f68c4; --m-main: #0b8a75; @@ -102,7 +241,11 @@ --m-db: #b5650d; --m-dup: #1f68c4; --m-pan: #a6338f; - --shadow: 0 1px 2px rgba(16, 27, 24, 0.06), 0 8px 24px rgba(16, 27, 24, 0.07); + --grid-minor: rgba(8, 92, 80, 0.026); + --grid-major: rgba(8, 92, 80, 0.058); + --tick: rgba(40, 70, 66, 0.3); + --tick-major: rgba(40, 70, 66, 0.6); + --corner: color-mix(in srgb, var(--accent) 80%, transparent); color-scheme: light; } @@ -114,66 +257,158 @@ html { scroll-behavior: smooth; - scroll-padding-top: 76px; + scroll-padding-top: var(--clear); -webkit-text-size-adjust: 100%; + /* The scrollbar's room is kept while the search sheet hides it, so the + page does not shift sideways under the sheet. */ + scrollbar-gutter: stable; +} + +/* The page under the open search sheet stays where it is; the sheet scrolls + on its own. */ +html.sheet-open { + overflow: hidden; } body { margin: 0; - background: var(--bg); - color: var(--ink); - font: 17px/1.55 var(--font); + color: var(--ink-1); + font: var(--fw-regular) var(--fs-0)/var(--lh-body) var(--font); -webkit-font-smoothing: antialiased; + -moz-osx-font-smoothing: grayscale; + font-kerning: normal; +} + +/* โ”€โ”€ The graticule โ”€โ”€ + The pages sit on a grid, the way a plot or an instrument's screen does: + minor lines every 16px, major ones every 80. It is the page's own + background, drawn from the top of the page and scrolled with it, so it + needs no layer of its own and nothing that moves or reflows moves it; + cards, plates, code and tables cover it. Its major lines fall on the + column's left edge, at every width: the 24px margin, 16px on a phone, and + from 1216px the edge of the centred column (half the screen less 584px; a + background's percentage is taken of the room left by its tile, hence the + two offsets). The docs keep only the major lines, quieter in the light + theme, so long reading stays calm. */ +:root { + --grid-x-major: 24px; + --grid-x-minor: 8px; +} + +@media (max-width: 600px) { + :root { + --grid-x-major: 16px; + --grid-x-minor: 0px; + } +} + +@media (min-width: 1216px) { + :root { + --grid-x-major: calc(50% - 544px); + --grid-x-minor: calc(50% - 576px); + --col-margin: calc(4 * var(--grid-sq) - var(--col-gap)); + } +} + +body { + background: + linear-gradient(90deg, var(--grid-major) 1px, transparent 1px) var(--grid-x-major) 0 / var(--grid-sq) var(--grid-sq), + linear-gradient(var(--grid-major) 1px, transparent 1px) 0 0 / var(--grid-sq) var(--grid-sq), + linear-gradient(90deg, var(--grid-minor) 1px, transparent 1px) var(--grid-x-minor) 0 / 16px 16px, + linear-gradient(var(--grid-minor) 1px, transparent 1px) 0 0 / 16px 16px, + var(--surface-0); +} + +body.docs { + --grid-minor: transparent; +} + +:root[data-theme="light"] body.docs { + --grid-major: rgba(8, 92, 80, 0.045); } +/* In the light theme the front page's chapters of reading (the steps, the + benchmark, the files, the questions) lay a veil of the page's colour over + the grid, from their own top rule: the lines go quieter there than behind + Figure 1 and the other drawings, and the rules of a table or a list stay + apart from them. */ +:root[data-theme="light"] :is(#start, #proof, #outputs, #faq) { + background-color: color-mix(in srgb, var(--surface-0) 50%, transparent); +} + +/* Links: accent ink, a quiet underline that takes the full colour on hover. */ a { color: var(--accent); + text-decoration-line: underline; text-decoration-thickness: 1px; - text-underline-offset: 3px; + text-decoration-color: color-mix(in srgb, currentColor 45%, transparent); + text-underline-offset: 0.22em; } -a:hover { - color: var(--accent-hover); +@media (hover: hover) { + a:hover { + color: var(--accent-hover); + text-decoration-color: currentColor; + } } :focus-visible { outline: 3px solid var(--accent); outline-offset: 2px; - border-radius: 4px; + border-radius: var(--r-1); } +::selection { + background: color-mix(in srgb, var(--accent) 32%, transparent); + color: var(--ink-1); +} + +/* Mono runs one step under the text around it (1/1.125). */ code, kbd, pre { font-family: var(--mono); - font-size: 0.88em; + font-size: 0.889em; + font-variant-ligatures: none; } +/* A chip of code never splits inside a word: short ones (.nw) do not wrap at + all, and the docs builder lets a long one wrap only after a /, _ or . + (see the Code section). */ :not(pre) > code { - background: var(--bg-subtle); - border: 1px solid var(--line); - border-radius: 6px; - padding: 0.08em 0.38em; - overflow-wrap: anywhere; + background: var(--surface-3); + border: 1px solid var(--line-2); + border-radius: var(--r-1); + padding: 0.06em 0.36em; + color: var(--ink-1); + overflow-wrap: normal; } +/* Headings: display (h1, h2) tight and heavy, the rest bold and a little + tight; all in the first ink. */ h1, h2, h3 { - line-height: 1.15; - letter-spacing: -0.02em; + line-height: var(--lh-heading); + letter-spacing: var(--tr-heading); margin: 0; + color: var(--ink-1); +} + +h1, +h2 { + font-weight: var(--fw-display); + letter-spacing: var(--tr-display); + line-height: var(--lh-display); } h2 { - font-size: clamp(1.6rem, 3.4vw, 2.2rem); - font-weight: 750; + font-size: var(--fs-title); } h3 { - font-size: 1.08rem; - font-weight: 700; - letter-spacing: -0.01em; + font-size: var(--fs-1); + font-weight: var(--fw-bold); } p { @@ -184,9 +419,9 @@ p { position: absolute; left: -999px; top: 8px; - background: var(--bg-elev); - padding: 8px 12px; - border-radius: 8px; + background: var(--surface-1); + padding: var(--sp-2) var(--sp-3); + border-radius: var(--r-3); z-index: 100; } @@ -195,7 +430,7 @@ p { } .wrap { - max-width: 1120px; + max-width: var(--wrap); margin: 0 auto; padding: 0 24px; } @@ -206,32 +441,319 @@ p { } } +/* โ”€โ”€ Surfaces โ”€โ”€ + Level 0 the page; 1 a card (its own fill and an edge, no shadow, 12px + corners); 2 a plate, what a showcase drawing stands on (a lighter fill, an + edge, a soft shadow, 16px corners); 3 an overlay (search, tips). Code sits + below the page. Inside a plate, controls rise one more level and the + drawings' halos take the plate's colour. */ +.demo, +.cases, +.case, +.metro-wrap, +.how-stage-card, +.excerpt, +.report-frame, +.lost { + --halo: var(--surface-2); + --control: var(--control-raised); + position: relative; +} + +/* A plate registers its frame on the graticule: its four corners drawn in + the accent, over its own edge and bent with it (its border, redrawn, and + masked down to the corners). The panels inside the front page's plate of + cases are not plates, until a phone deals them out as cards. */ +:is(.demo, .cases, .case, .metro-wrap, .how-stage-card, .excerpt, .report-frame, .lost)::after { + --c: 22px; + content: ""; + position: absolute; + inset: -1px; + border: 1px solid var(--corner); + border-radius: inherit; + pointer-events: none; + -webkit-mask: + linear-gradient(#000 0 0) 0 0 / var(--c) var(--c) no-repeat, + linear-gradient(#000 0 0) 100% 0 / var(--c) var(--c) no-repeat, + linear-gradient(#000 0 0) 0 100% / var(--c) var(--c) no-repeat, + linear-gradient(#000 0 0) 100% 100% / var(--c) var(--c) no-repeat; + mask: + linear-gradient(#000 0 0) 0 0 / var(--c) var(--c) no-repeat, + linear-gradient(#000 0 0) 100% 0 / var(--c) var(--c) no-repeat, + linear-gradient(#000 0 0) 0 100% / var(--c) var(--c) no-repeat, + linear-gradient(#000 0 0) 100% 100% / var(--c) var(--c) no-repeat; +} + +.cases > .case::after { + content: none; +} + +/* Headings and short labels break into lines of even length rather than + leave one word; running text and the texts of cards (spans inside links, + which p and li do not reach) keep their last line from being one word. */ +h1, +h2, +h3, +h4, +summary, +.eyebrow, +.fig-title, +.cmd-head, +.how-steps button, +.deep a b, +.docs-nav .page-toc a, +.toc-inline a, +.pf-headrow span, +.pf-name b, +.pr-step b, +.opt b, +.how-genome { + text-wrap: balance; +} + +p, +li, +dd, +small, +blockquote, +figcaption, +.deep a span, +.rp-text, +.oc-text { + text-wrap: pretty; +} + +/* A control's label is not part of the text: selecting across it neither + paints it unreadable on an accent fill nor copies "Copy" with the rest. */ +.btn, +.mini-btn, +.copy, +.tab, +.seg button, +.how-group .seg a, +.how-steps button, +.hero-seg, +.search-btn, +.sheet-close, +.icon-btn, +.gh-link, +.report-cover { + -webkit-user-select: none; + user-select: none; +} + +/* Read by screen readers, not drawn. */ +.visually-hidden { + position: absolute !important; + width: 1px; + height: 1px; + margin: -1px; + padding: 0; + overflow: hidden; + clip: rect(0 0 0 0); + clip-path: inset(50%); + white-space: nowrap; + border: 0; +} + +/* The theme switch cross-fades where the browser can (site.js skips it under + reduced motion). */ +::view-transition-old(root), +::view-transition-new(root) { + animation-duration: var(--dur-state); +} + +/* For the frames of the swap itself, no control fades on its own from the + old theme's colours (site.js sets the class and takes it off). */ +.theme-swap *, +.theme-swap *::before, +.theme-swap *::after { + transition: none !important; +} + +/* Every control changes state at one speed: colour, fill, edge, shadow and + fade together. A press (the 1px sink) is not animated. */ +.nav a, +.gh-link, +.icon-btn, +.search-btn, +.btn, +.mini-btn, +.tab, +.seg button, +.copy, +.opt, +.metro-opts label, +.docs-nav li a, +.sheet-links a, +.faq summary, +.how-group .seg a, +.how-steps button, +.prose .anchor, +.sheet-close, +.slash-toggle, +.hero-seg::before, +.hero-seg i { + transition: color var(--dur-state) var(--ease), background-color var(--dur-state) var(--ease), border-color var(--dur-state) var(--ease), box-shadow var(--dur-state) var(--ease), opacity var(--dur-state) var(--ease); +} + +/* "Link copied" and "Copied", beside what was copied (placed by site.js). */ +.copied-note { + position: absolute; + z-index: 60; + transform: translateY(-50%); + padding: var(--sp-1) var(--sp-2); + border: 1px solid color-mix(in srgb, var(--accent) 45%, transparent); + border-radius: var(--r-2); + background: var(--accent-soft); + color: var(--accent); + font: var(--fw-semibold) var(--fs-n2)/var(--lh-label) var(--font); + letter-spacing: 0; + white-space: nowrap; + pointer-events: none; + animation: note-in 0.15s ease-out; +} + +.copied-note.above { + transform: translateY(-100%); +} + +@keyframes note-in { + from { + opacity: 0; + translate: 0 3px; + } +} + +.copyable { + cursor: copy; +} + +@media (hover: hover) { + .copyable:hover { + color: var(--accent); + border-color: var(--accent); + } +} + +/* The heading a link lands on: an accent bar in the margin that fades. */ +.arrive { + position: relative; +} + +.arrive::before { + content: ""; + position: absolute; + left: -14px; + top: 0.1em; + bottom: 0.1em; + width: 4px; + border-radius: 2px; + background: var(--accent); + pointer-events: none; + animation: arrive 1.2s ease-out forwards; +} + +@keyframes arrive { + 0%, + 45% { + opacity: 1; + } + 100% { + opacity: 0; + } +} + +/* The words of the search that brought the reader to this section. */ +mark.search-hit { + background: color-mix(in srgb, var(--accent) 24%, transparent); + color: inherit; + border-radius: 3px; + box-shadow: 0 0 0 1px color-mix(in srgb, var(--accent) 24%, transparent); +} + +@media (prefers-reduced-motion: reduce) { + .copied-note { + animation: none; + } +} + +/* More contrast on request: quieter text and edges step up. */ +@media (prefers-contrast: more) { + :root { + --ink-3: #b4c0bd; + --line-2: #56645f; + --line-3: #7d8b87; + --grid-minor: transparent; + --grid-major: rgba(104, 233, 212, 0.05); + } + + :root[data-theme="light"] { + --ink-3: #3f4b48; + --line-2: #93a09c; + --line-3: #66736f; + --grid-minor: transparent; + } +} + /* โ”€โ”€ Header โ”€โ”€ */ .top { position: sticky; top: 0; z-index: 50; - background: color-mix(in srgb, var(--bg) 86%, transparent); - -webkit-backdrop-filter: saturate(1.4) blur(10px); - backdrop-filter: saturate(1.4) blur(10px); - border-bottom: 1px solid var(--line); + background: color-mix(in srgb, var(--surface-0) 80%, transparent); + -webkit-backdrop-filter: saturate(1.6) blur(14px); + backdrop-filter: saturate(1.6) blur(14px); + border-bottom: 1px solid var(--line-1); +} + +/* Along the header's lower edge, a ruler on the graticule: a tick on every + minor line, a taller one on every major. */ +.top::after { + content: ""; + position: absolute; + left: 0; + right: 0; + bottom: 0; + height: 7px; + pointer-events: none; + background: + linear-gradient(90deg, var(--tick-major) 1px, transparent 1px) var(--grid-x-major) 100% / var(--grid-sq) 7px repeat-x, + linear-gradient(90deg, var(--tick) 1px, transparent 1px) var(--grid-x-minor) 100% / 16px 3px repeat-x; } .top .wrap { display: flex; align-items: center; - gap: 16px; - height: 60px; + gap: var(--sp-4); + height: calc(var(--header-h) - 1px); } .brand { display: inline-flex; align-items: center; - gap: 10px; + gap: var(--sp-3); flex: none; text-decoration: none; - color: var(--ink); + color: var(--ink-1); +} + +@media (hover: hover) { + .brand:hover { + color: var(--ink-1); + } +} + +/* The logo is drawn for a dark ground; the light theme takes the variant + made from it (same shapes, its greys turned to ink and its teals deepened). */ +.brand .logo-light, +:root[data-theme="light"] .brand .logo-dark { + display: none; +} + +:root[data-theme="light"] .brand .logo-light { + display: block; } .brand img { @@ -241,37 +763,67 @@ p { } .brand span { - font-weight: 750; + font-weight: var(--fw-display); letter-spacing: -0.01em; - font-size: 1.02rem; + font-size: var(--fs-0); } -@media (max-width: 520px) { +/* The logo already says AMAP: its text is the first thing to go when the + header runs short, before any control does. */ +@media (max-width: 699px) { .brand span { display: none; } } +/* The links, then the controls, 4px apart; the controls stand 8px further + off the last link (Docs, on every page), so the two groups read as two. + Links and buttons are one height, 36px. */ .nav { display: flex; - gap: 2px; + column-gap: var(--sp-1); margin-left: auto; align-items: center; + min-width: 0; } .nav a { - color: var(--ink-soft); + display: inline-flex; + align-items: center; + min-height: 36px; + color: var(--ink-2); text-decoration: none; - font-size: 0.93rem; - padding: 7px 10px; - border-radius: 8px; + font-size: var(--fs-n1); + font-weight: var(--fw-medium); + padding: 0 10px; + border-radius: var(--r-3); white-space: nowrap; } -.nav a:hover, -.nav a[aria-current="page"] { - color: var(--ink); - background: var(--bg-subtle); +.nav a[href$="documentation.html"] { + margin-right: var(--sp-2); +} + +@media (hover: hover) { + .nav a:hover { + color: var(--ink-1); + background: var(--surface-3); + } +} + +/* Where you are, apart from where the pointer is: accent text on an accent + bar. */ +.nav a[aria-current] { + color: var(--accent); + font-weight: var(--fw-semibold); + box-shadow: inset 0 -2px 0 var(--accent); + border-radius: var(--r-3) var(--r-3) 0 0; +} + +@media (hover: hover) { + .nav a[aria-current]:hover { + color: var(--accent-hover); + } } .gh-link { @@ -280,20 +832,22 @@ p { gap: 7px; height: 36px; padding: 0 12px !important; - margin-left: 4px; - border: 1px solid var(--line); - border-radius: 9px; - background: var(--bg-elev); - color: var(--ink-soft); - font-size: 0.9rem; - font-weight: 600; + border: 1px solid var(--line-2); + border-radius: var(--r-3); + background: var(--control); + color: var(--ink-1); + font-size: var(--fs-n2); + font-weight: var(--fw-semibold); text-decoration: none; flex: none; } -.gh-link:hover { - color: var(--ink); - border-color: var(--line-strong); +@media (hover: hover) { + .gh-link:hover { + color: var(--ink-1); + border-color: var(--line-3); + background: color-mix(in srgb, var(--ink-1) 7%, var(--control)); + } } .gh-link svg { @@ -301,29 +855,25 @@ p { height: 17px; } -@media (max-width: 420px) { - .gh-link span { - display: none; - } -} - .icon-btn { display: inline-grid; place-items: center; width: 36px; height: 36px; - margin-left: 4px; - border: 1px solid var(--line); - border-radius: 9px; - background: var(--bg-elev); - color: var(--ink-soft); + border: 1px solid var(--line-2); + border-radius: var(--r-3); + background: var(--control); + color: var(--ink-2); cursor: pointer; flex: none; } -.icon-btn:hover { - color: var(--ink); - border-color: var(--line-strong); +@media (hover: hover) { + .icon-btn:hover { + color: var(--ink-1); + border-color: var(--line-3); + background: color-mix(in srgb, var(--ink-1) 7%, var(--control)); + } } .icon-btn svg { @@ -331,2134 +881,5646 @@ p { height: 18px; } -@media (max-width: 860px) { - .nav .nav-optional { - display: none; - } +/* The theme toggle shows what a press gives: a sun on the dark theme, a moon + on the light one (site.js adds the moon). */ +[data-theme-toggle] .i-moon, +:root[data-theme="light"] [data-theme-toggle] .i-sun { + display: none; } -@media (max-width: 380px) { - .nav a { - padding: 7px 7px; - } +:root[data-theme="light"] [data-theme-toggle] .i-moon { + display: block; } -/* โ”€โ”€ Buttons โ”€โ”€ */ - -.btn { +/* Search, added to every page's header by site.js: a field-like button with + its "/" key on wide screens, a square icon below. */ +.search-btn { display: inline-flex; align-items: center; gap: 8px; - padding: 11px 18px; - border-radius: 10px; - font-weight: 650; - font-size: 0.98rem; - text-decoration: none; - border: 1px solid transparent; + height: 36px; + min-width: 36px; + padding: 0 7px 0 11px; + border: 1px solid var(--line-input); + border-radius: var(--r-3); + background: var(--control); + color: var(--muted); + font: var(--fs-n2) var(--font); cursor: pointer; - line-height: 1.2; - font-family: var(--font); -} - -.btn svg { - width: 17px; - height: 17px; flex: none; } -.btn-primary { - background: var(--accent); - color: var(--on-accent); -} - -.btn-primary:hover { - background: var(--accent-hover); - color: var(--on-accent); -} - -.btn-ghost { - background: var(--bg-elev); - color: var(--ink); - border-color: var(--line-strong); +@media (hover: hover) { + .search-btn:hover, + .docs-search input:hover { + color: var(--ink); + border-color: var(--ink-3); + } } -.btn-ghost:hover { - color: var(--ink); - border-color: var(--ink-soft); +.search-btn svg { + width: 16px; + height: 16px; + flex: none; } -.mini-btn { - border: 1px solid var(--line-strong); - background: var(--bg-elev); - color: var(--ink-soft); - font: 600 0.8rem var(--font); - padding: 5px 10px; - border-radius: 7px; - cursor: pointer; +.search-btn-label { + padding-right: 14px; } -.mini-btn:hover { - color: var(--ink); - border-color: var(--ink-soft); +.search-btn[aria-expanded="true"] { + color: var(--accent); + border-color: var(--accent); } -/* โ”€โ”€ Hero โ”€โ”€ */ +@media (max-width: 899px) { + .search-btn { + justify-content: center; + padding: 0; + border-color: var(--line-2); + color: var(--ink-soft); + } -.hero { - padding: 60px 0 44px; + .search-btn .search-btn-label, + .search-btn .search-key { + display: none; + } } -.hero .wrap { - display: grid; - grid-template-columns: minmax(0, 0.95fr) minmax(0, 1.1fr); - gap: 44px; - align-items: center; +/* Optional links leave the header one at a time, as room runs out; the + search sheet lists them too. */ +@media (max-width: 639px) { + .nav a.nav-optional[href$="tutorial.html"] { + display: none; + } } -@media (max-width: 940px) { - .hero { - padding-top: 36px; +@media (max-width: 559px) { + .nav a.nav-optional[href$="how-it-works.html"] { + display: none; } +} - .hero .wrap { - grid-template-columns: minmax(0, 1fr); - gap: 32px; +/* A page whose own link has left the bar (the tutorial's under 640px, How + it works' under 560px) marks Docs, the link that stays for it, so the bar + still says where the reader is. */ +@media (max-width: 639px) { + .nav:has(> a.nav-optional[href$="tutorial.html"][aria-current]) > a[href$="documentation.html"] { + color: var(--accent); + font-weight: var(--fw-semibold); + box-shadow: inset 0 -2px 0 var(--accent); + border-radius: var(--r-3) var(--r-3) 0 0; } } -.eyebrow { - display: inline-flex; - align-items: center; - gap: 8px; - font-size: 0.85rem; - font-weight: 650; - color: var(--muted); - margin-bottom: 18px; +@media (max-width: 559px) { + .nav:has(> a.nav-optional[href$="how-it-works.html"][aria-current]) > a[href$="documentation.html"] { + color: var(--accent); + font-weight: var(--fw-semibold); + box-shadow: inset 0 -2px 0 var(--accent); + border-radius: var(--r-3) var(--r-3) 0 0; + } } -.eyebrow .dot { - width: 8px; - height: 8px; - border-radius: 50%; - background: linear-gradient(135deg, var(--brand-aqua), var(--brand-slate)); - flex: none; +.top.squeeze-3 .nav:has(> a.nav-optional[href$="tutorial.html"][aria-current]) > a[href$="documentation.html"], +.top.squeeze-4 .nav:has(> a.nav-optional[href$="how-it-works.html"][aria-current]) > a[href$="documentation.html"] { + color: var(--accent); + font-weight: var(--fw-semibold); + box-shadow: inset 0 -2px 0 var(--accent); + border-radius: var(--r-3) var(--r-3) 0 0; } -.hero h1 { - font-size: clamp(2.2rem, 5vw, 3.6rem); - font-weight: 800; - letter-spacing: -0.035em; - line-height: 1.04; -} +/* On a phone the controls keep their room: tighter gaps, a smaller logo, and + GitHub as its mark (its name stays for screen readers). */ +@media (max-width: 439px) { + .top .wrap { + gap: 8px; + } -.grad { - background: linear-gradient(100deg, var(--grad-a) 0%, var(--grad-b) 55%, var(--grad-c) 100%); - -webkit-background-clip: text; - background-clip: text; - color: transparent; -} + .brand img { + height: 34px; + } -.lead { - font-size: 1.14rem; - color: var(--ink-soft); - margin-top: 20px; - max-width: 34em; -} + .nav { + column-gap: 2px; + } -.actions { - display: flex; - flex-wrap: wrap; - gap: 12px; - margin-top: 28px; -} + .nav a { + padding: 0 8px; + } -.reads { - margin-top: 22px; - font-size: 0.9rem; - color: var(--muted); - max-width: 36em; -} + .nav a[href$="documentation.html"] { + margin-right: var(--sp-1); + } -.reads b { - color: var(--ink-soft); - font-weight: 600; -} + /* GitHub as its mark alone: a square, as the other two. */ + .gh-link { + padding: 0 !important; + width: 36px; + justify-content: center; + } -/* โ”€โ”€ Gene tracks: the drawings โ”€โ”€ */ + .gh-link span { + position: absolute; + width: 1px; + height: 1px; + overflow: hidden; + clip-path: inset(50%); + white-space: nowrap; + } +} -/* Every drawing is an SVG of rows on one coordinate axis. The colours are - classes, so a drawing follows the theme like the rest of the page. */ -.trk { - display: block; - width: 100%; - height: auto; - overflow: visible; - font-family: var(--font); +@media (max-width: 380px) { + .nav a { + padding: 0 6px; + } } -.trk text { - fill: var(--ink-soft); - font-size: 12px; +/* Under 360px GitHub leaves the bar, so the links keep their spacing and + the buttons a finger's 40px; the page's foot links to it. */ +@media (max-width: 359px) { + .nav .gh-link { + display: none; + } } -.trk .row-label { - fill: var(--muted); - font-size: 11px; - font-weight: 650; - letter-spacing: 0.02em; +/* The widths above are measured with the usual system fonts. When the row + still runs over (a wider font, larger text, extra letter spacing), site.js + sheds the same items in this order, one step at a time, until it fits. */ +.top.squeeze-1 .brand span { + display: none; } -.trk .row-line { - stroke: var(--line-strong); - stroke-width: 1; +.top.squeeze-2 .search-btn { + justify-content: center; + padding: 0; + border-color: var(--line-2); } -.trk .axis line, -.trk .axis path { - stroke: var(--line-strong); +.top.squeeze-2 .search-btn .search-btn-label, +.top.squeeze-2 .search-btn .search-key, +.top.squeeze-3 .nav a.nav-optional[href$="tutorial.html"], +.top.squeeze-4 .nav a.nav-optional[href$="how-it-works.html"], +.top.squeeze-6 .nav .gh-link { + display: none; } -.trk .axis text { - fill: var(--muted); - font-size: 10px; - font-family: var(--mono); +.top.squeeze-5 .gh-link { + padding: 0 !important; + width: 36px; + justify-content: center; } -.trk .g { - stroke-width: 1.4; - transition: opacity 0.35s ease, transform 0.45s ease; +.top.squeeze-5 .gh-link span { + position: absolute; + width: 1px; + height: 1px; + overflow: hidden; + clip-path: inset(50%); + white-space: nowrap; } -.trk .g-bakta { - fill: color-mix(in srgb, var(--bakta) 26%, transparent); - stroke: var(--bakta); +/* โ”€โ”€ Buttons โ”€โ”€ */ + +/* One button, three looks: primary (the accent, one per view), secondary + (a card's surface and edge) and the small one inside drawings and code. + Heights sit on the 4px grid: 44, 36 and 28. */ +.btn { + display: inline-flex; + align-items: center; + justify-content: center; + gap: var(--sp-2); + min-height: 44px; + padding: 0 var(--sp-5); + border-radius: var(--r-3); + font-weight: var(--fw-semibold); + font-size: var(--fs-n1); + letter-spacing: -0.005em; + text-decoration: none; + border: 1px solid transparent; + cursor: pointer; + line-height: 1.2; + font-family: var(--font); } -.trk .g-ref { - fill: color-mix(in srgb, var(--ref) 26%, transparent); - stroke: var(--ref); +.btn svg { + width: 17px; + height: 17px; + flex: none; } -.trk .g-amap { - fill: var(--amap); - stroke: var(--amap); +.btn-primary { + background: var(--accent); + color: var(--on-accent); + box-shadow: var(--glow); } -.trk .g-drop { - fill: transparent; - stroke: var(--drop); - stroke-dasharray: 4 3; +@media (hover: hover) { + .btn-primary:hover { + background: var(--accent-hover); + color: var(--on-accent); + } } -.trk .g-pseudo { - fill: url(#hatch); - stroke: var(--pseudo); +.btn-ghost { + background: var(--control); + color: var(--ink-1); + border-color: var(--line-2); } -.trk .hatch-line { - stroke: var(--pseudo); - stroke-width: 2; +@media (hover: hover) { + .btn-ghost:hover { + color: var(--ink-1); + background: color-mix(in srgb, var(--ink-1) 7%, var(--control)); + border-color: var(--line-3); + } } -.trk .gname { - font-size: 11.5px; - font-weight: 650; - fill: var(--ink); +.mini-btn { + min-height: 28px; + border: 1px solid var(--line-2); + background: var(--control); + color: var(--ink-1); + font: var(--fw-semibold) var(--fs-n2) var(--font); + padding: 0 var(--sp-3); + border-radius: var(--r-2); + cursor: pointer; } -.trk .gname-on { - fill: var(--on-track); +@media (hover: hover) { + .mini-btn:hover { + color: var(--ink-1); + border-color: var(--line-3); + background: color-mix(in srgb, var(--ink-1) 7%, var(--control)); + } } -.trk .gsub { - font-size: 10px; - fill: var(--muted); +/* โ”€โ”€ Search sheet โ”€โ”€ */ + +/* Under the sticky header, the same box and results on every page; on a + phone it also holds the page's contents and the pages people look for. */ +.search-sheet { + position: fixed; + top: var(--header-h); + left: 0; + right: 0; + z-index: 49; + max-height: calc(100vh - var(--header-h)); + max-height: calc(100dvh - var(--header-h)); + overflow: auto; + overscroll-behavior: contain; + padding: var(--sp-4) 0 var(--sp-6); + background: var(--surface-1); + border-bottom: 1px solid var(--line-2); + box-shadow: var(--shadow-3); } -.trk .gap-mark { - stroke: var(--ref); - stroke-width: 2; +.search-sheet[hidden], +.search-scrim[hidden] { + display: none; } -.trk .diff { - fill: color-mix(in srgb, var(--warn) 22%, transparent); - stroke: none; +.search-scrim { + position: fixed; + inset: var(--header-h) 0 0; + z-index: 48; + background: color-mix(in srgb, var(--bg) 62%, transparent); } -.trk .diff-ref { - fill: color-mix(in srgb, var(--ref) 18%, transparent); +.search-sheet .wrap { + max-width: 760px; } -.trk .diff-label.diff-label-ref { - fill: var(--ref); +.sheet-search { + margin: 0; } -.trk .g-cut { - stroke-dasharray: none; +.sheet-search > svg { + position: absolute; + left: 14px; + top: 15px; + width: 17px; + height: 17px; + color: var(--muted); + pointer-events: none; } -.trk .diff-line { - stroke: var(--warn); - stroke-width: 1; - stroke-dasharray: 2 2; +.search-sheet .sheet-search input { + height: 48px; + padding: 0 70px 0 42px; + font-size: var(--fs-n1); + border-radius: var(--r-4); + background: var(--bg); } -.trk .diff-label { - fill: var(--warn); - font-size: 10.5px; - font-weight: 700; +.sheet-close { + position: absolute; + right: 8px; + top: 8px; + height: 32px; + padding: 0 var(--sp-3); + border: 1px solid var(--line-strong); + border-radius: var(--r-2); + background: var(--control); + color: var(--muted); + font: var(--fw-semibold) var(--fs-n2) var(--font); + cursor: pointer; } -.trk .edge { - stroke: var(--ink-soft); - stroke-width: 2; +@media (hover: hover) { + .sheet-close:hover { + color: var(--ink-1); + border-color: var(--line-3); + background: color-mix(in srgb, var(--ink-1) 7%, var(--control)); + } } -.trk .edge-label { - fill: var(--muted); - font-size: 10px; +/* "Esc" where there is a keyboard to press it on, "Close" on a touch screen. */ +.sheet-close .on-touch { + display: none; } -.trk .badge rect { - rx: 8; +@media (hover: none) { + .sheet-close .on-keys { + display: none; + } + + .sheet-close .on-touch { + display: inline; + } } -.trk .badge text { - font-size: 10px; - font-weight: 750; - text-anchor: middle; - fill: var(--on-track); +/* In the sheet the results are part of the sheet, not a dropdown. */ +.search-sheet .docs-results { + position: static; + margin-top: 12px; + padding: 0; + border: 0; + border-radius: 0; + background: none; + box-shadow: none; + max-height: none; + overflow: visible; } -.trk .b-case1 rect { - fill: var(--muted); +.search-sheet .docs-results .r-count { + position: static; + margin: 0 0 4px; + padding: 0 10px 8px; + background: none; } -.trk .b-case2 rect, -.trk .b-case3 rect { - fill: var(--ok); +.search-sheet .docs-results .r-foot { + position: static; + margin: 8px 0 0; + background: none; } -.trk .b-case4 rect { - fill: var(--warn); +.search-sheet.has-query .sheet-empty { + display: none; } -.trk .b-miss rect { - fill: var(--ref); +.sheet-empty { + display: grid; + gap: 20px; + margin-top: 20px; } -.trk .rib { - stroke: none; - transition: opacity 0.45s ease; +.sheet-label { + margin: 0 0 8px; + color: var(--muted); + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + letter-spacing: var(--tr-caps); + text-transform: uppercase; } -.trk .rib-agree { - fill: color-mix(in srgb, var(--ok) 13%, transparent); +.sheet-toc ol, +.sheet-links { + list-style: none; + margin: 0; + padding: 0; } -.trk .rib-conflict { - fill: color-mix(in srgb, var(--warn) 22%, transparent); +.sheet-toc a { + display: block; + padding: 8px 12px; + border-left: 2px solid var(--line); + color: var(--ink-soft); + text-decoration: none; + line-height: 1.35; } -.trk .hot { - transition: opacity 0.45s ease; +@media (hover: hover) { + .sheet-toc a:hover { + color: var(--ink); + } } -/* A call nothing aligns to, once the calls are compared. */ -[data-phase="3"] .c1-call .g, -[data-phase="4"] .c1-call .g { - stroke-dasharray: 3 2; - stroke: var(--drop); - fill: transparent; +.sheet-toc a.active { + border-left-color: var(--accent); + background: var(--accent-soft); + color: var(--ink); + font-weight: var(--fw-semibold); } -/* โ”€โ”€ The animated stretch of genome in the hero โ”€โ”€ */ +.sheet-links { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(160px, 1fr)); + gap: 8px; +} -.demo { - background: var(--bg-elev); +.sheet-links a { + display: block; + height: 100%; + padding: var(--sp-3); border: 1px solid var(--line); - border-radius: 16px; - padding: 18px 18px 14px; - box-shadow: var(--shadow); - margin: 0; + border-radius: var(--r-3); + background: var(--bg); + color: var(--ink); + text-decoration: none; } -.demo-head { - display: flex; - justify-content: space-between; - align-items: flex-start; - gap: 12px; - margin-bottom: 8px; - font-size: 0.86rem; - color: var(--muted); +@media (hover: hover) { + .sheet-links a:hover { + border-color: var(--accent-line); + background: var(--surface-2); + } } -.demo-head b { - color: var(--ink); - font-weight: 650; +.sheet-links b { + display: block; + font-size: var(--fs-n1); } -.demo-ctrl { - display: inline-flex; - align-items: center; - gap: 10px; - flex: none; +.sheet-links small { + display: block; + margin-top: 1px; + color: var(--muted); + font-size: var(--fs-n2); } -.demo-step { - color: var(--ink); - font-weight: 650; - white-space: nowrap; +.sheet-links .sheet-last { + border-style: dashed; } -.demo-stage { - position: relative; +/* On a phone, a slim bar under the header names the section being read and + draws how far down the page you are; a tap opens the sheet. */ +.here-bar { + display: none; } -.demo-tip { - position: absolute; - z-index: 5; - max-width: 290px; - padding: 9px 11px; - border-radius: 9px; - background: var(--bg); - border: 1px solid var(--line-strong); - box-shadow: var(--shadow); - font-size: 0.82rem; - line-height: 1.4; - color: var(--ink-soft); - pointer-events: none; -} +@media (max-width: 860px) { + html.has-here-bar { + scroll-padding-top: calc(var(--header-h) + var(--here-h) + var(--sp-4) + 1px); + } -.demo-tip b { - color: var(--ink); -} + .here-bar { + position: fixed; + top: var(--header-h); + left: 0; + right: 0; + z-index: 47; + display: flex; + align-items: center; + gap: 8px; + height: var(--here-h); + padding: 0 16px; + border: 0; + border-bottom: 1px solid var(--line); + background: color-mix(in srgb, var(--bg) 90%, transparent); + -webkit-backdrop-filter: saturate(1.4) blur(10px); + backdrop-filter: saturate(1.4) blur(10px); + color: var(--ink-soft); + font: var(--fw-semibold) var(--fs-n2) var(--font); + text-align: left; + cursor: pointer; + transform: translateY(-100%); + opacity: 0; + visibility: hidden; + transition: transform 0.2s ease, opacity 0.2s ease, visibility 0s linear 0.2s; + } -.demo-tip code { - font-size: 0.78rem; -} + .here-bar.on { + transform: none; + opacity: 1; + visibility: visible; + transition-delay: 0s; + } -.legend { - display: flex; - flex-wrap: wrap; - gap: 6px 16px; - margin-top: 10px; - font-size: 0.8rem; - color: var(--muted); -} + .sheet-open .here-bar { + visibility: hidden; + } -.legend span { - display: inline-flex; - align-items: center; - gap: 6px; -} + /* The ring inside the bar: outside, the header and the screen's edges + would cut it to a line that reads as the progress bar. */ + .here-bar:focus-visible { + outline-offset: -3px; + } -.chip { - width: 18px; - height: 10px; - border-radius: 3px; - display: inline-block; - border: 1.5px solid; - flex: none; -} + .here-label { + flex: 1; + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; + } -.chip.bakta { - border-color: var(--bakta); - background: color-mix(in srgb, var(--bakta) 26%, transparent); -} + .here-bar svg { + width: 16px; + height: 16px; + flex: none; + color: var(--muted); + } -.chip.ref { - border-color: var(--ref); - background: color-mix(in srgb, var(--ref) 26%, transparent); + .here-progress { + position: absolute; + left: 0; + right: 0; + bottom: -1px; + height: 2px; + background: var(--accent); + transform: scaleX(0); + transform-origin: 0 50%; + } } -.chip.amap { - border-color: var(--amap); - background: var(--amap); -} +/* โ”€โ”€ Touch and pointer โ”€โ”€ */ -.chip.drop { - border-color: var(--drop); - border-style: dashed; - background: transparent; +/* A press is drawn by the design, not by the browser's grey flash (the + colour is inherited, so the genes and marks of a drawing lose it too): + buttons sink a pixel and take a firmer fill. */ +html { + -webkit-tap-highlight-color: transparent; } -.chip.band-agree { - border: 0; - background: color-mix(in srgb, var(--ok) 30%, transparent); +.btn:active, +.mini-btn:active, +.tab:active, +.copy:active, +.icon-btn:active, +.gh-link:active, +.search-btn:active, +.sheet-close:active, +.seg button:active, +.how-group .seg a:active, +.how-steps button:active { + transform: translateY(1px); } -.chip.band-conflict { - border: 0; - background: color-mix(in srgb, var(--warn) 40%, transparent); +/* A button that cannot be pressed does not sink. */ +.btn[aria-disabled="true"]:active { + transform: none; } -.chip.pseudo { - border-color: var(--pseudo); - background: repeating-linear-gradient(135deg, var(--pseudo) 0 2px, transparent 2px 5px); +.btn-primary:active { + background: color-mix(in srgb, var(--accent) 78%, #000); } -.tally { - display: grid; - grid-template-columns: repeat(3, minmax(0, 1fr)); - gap: 10px; - margin-top: 12px; - padding-top: 12px; - border-top: 1px solid var(--line); - font-size: 0.82rem; - color: var(--muted); +.btn-ghost:active, +.mini-btn:active, +.tab:active, +.copy:active, +.icon-btn:active, +.gh-link:active, +.search-btn:active, +.sheet-close:active, +.seg button:active, +.how-group .seg a:active, +.how-steps button:active { + background: color-mix(in srgb, var(--ink) 12%, var(--bg-elev)); } -.tally b { - display: block; - font-size: 1.35rem; - line-height: 1.1; - color: var(--ink); - font-variant-numeric: tabular-nums; +.nav a:active, +.docs-nav li a:active, +.here-bar:active { + background: var(--bg-subtle); } -.tally .t-agree b { - color: var(--ok); -} +/* A finger's 40px targets are set at the end of the sheet (Touch, again), + after the rules of the controls they enlarge. */ -.tally .t-fixed b { - color: var(--warn); -} +@media (prefers-reduced-motion: reduce) { + .btn:active, + .mini-btn:active, + .tab:active, + .copy:active, + .icon-btn:active, + .gh-link:active, + .search-btn:active, + .sheet-close:active, + .seg button:active, + .how-group .seg a:active, + .how-steps button:active { + transform: none; + } -.tally .t-drop b { - color: var(--muted); + .here-bar { + transition: none !important; + } } -/* Before the calls are compared, the counts wait in grey, still readable. */ -.tally .t-off b { - color: var(--muted); +/* โ”€โ”€ Hero โ”€โ”€ */ + +/* The statement spans the column; under it the words take the left and what + goes in, what comes out and the way in take the right. Then the drawing, + the whole width of the column: Figure 1, every gene of it named, its + counts read out beside it. On a 1280 by 800 laptop all of it is in the + first screen. A faint wash of the accent sits behind the drawing. */ +.hero { + position: relative; + padding: var(--sp-10) 0 var(--sp-11); + overflow: clip; } -.tally b { - transition: color 0.4s ease; +.hero::before { + content: ""; + position: absolute; + inset: 0; + background: + radial-gradient(48% 46% at 72% 78%, color-mix(in srgb, var(--accent) 9%, transparent), transparent 70%), + radial-gradient(30% 40% at 8% 0%, color-mix(in srgb, var(--brand-slate) 6%, transparent), transparent 70%); + pointer-events: none; } -/* The phases: each class reveals one more layer. */ -[data-phase] .ph-ref, -[data-phase] .ph-case, -[data-phase] .ph-amap { - opacity: 0; - transition: opacity 0.45s ease; +:root[data-theme="light"] .hero::before { + background: radial-gradient(48% 46% at 72% 78%, color-mix(in srgb, var(--accent) 7%, transparent), transparent 70%); } -[data-phase="2"] .ph-ref, -[data-phase="3"] .ph-ref, -[data-phase="4"] .ph-ref, -[data-phase="3"] .ph-case, -[data-phase="4"] .ph-case, -[data-phase="4"] .ph-amap { - opacity: 1; +.hero .wrap { + position: relative; + display: grid; + grid-template-columns: minmax(0, 7fr) minmax(0, 5fr); + grid-template-areas: + "eyebrow eyebrow" + "title title" + "lead side" + "demo demo"; + column-gap: var(--sp-10); + align-items: start; } -[data-phase="4"] .ph-dim { - opacity: 0.35; +.hero-text { + display: contents; } -.trk .hot { - cursor: pointer; +.hero .eyebrow { + grid-area: eyebrow; } -.trk .hot:hover .g, -.trk .hot:focus .g, -.trk .hot.on .g { - stroke-width: 2.6; +.hero h1 { + grid-area: title; } -.trk .hot:focus { - outline: none; +.hero .lead { + grid-area: lead; } -/* โ”€โ”€ Sections โ”€โ”€ */ +.hero-side { + grid-area: side; + display: grid; + gap: var(--sp-6); + align-content: start; + padding-top: var(--sp-1); +} -.section { - padding: 72px 0; - border-top: 1px solid var(--line); +.hero .demo { + grid-area: demo; + margin-top: var(--sp-8); } -@media (max-width: 600px) { - .section { - padding: 52px 0; +/* The header has GitHub; the hero's own button goes first when room runs + short, so the other two stay on one line. */ +@media (max-width: 1100px) { + .hero .btn-gh { + display: none; } } -.section-head { - margin-bottom: 30px; - max-width: 780px; -} +/* One column: the words, the way in, the drawing, then the spec. */ +@media (max-width: 940px) { + .hero { + padding: var(--sp-9) 0 var(--sp-10); + } -.kicker { - font-size: 0.8rem; - font-weight: 750; - text-transform: uppercase; - letter-spacing: 0.08em; - color: var(--accent); - margin-bottom: 10px; -} + .hero .wrap { + grid-template-columns: minmax(0, 1fr); + grid-template-areas: "eyebrow" "title" "lead" "actions" "demo" "spec"; + } -.section-head p { - color: var(--muted); - font-size: 1.04rem; - margin-top: 10px; -} + .hero-side { + display: contents; + } -.cols-2 { - display: grid; - grid-template-columns: repeat(2, minmax(0, 1fr)); - gap: 20px; -} + .hero .actions { + grid-area: actions; + } -@media (max-width: 820px) { - .cols-2 { - grid-template-columns: minmax(0, 1fr); + .hero .hero-side .actions { + margin-top: var(--sp-7); + } + + .hero .spec { + grid-area: spec; + margin-top: var(--sp-7); } } -.card { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: var(--radius); - padding: 22px; +/* The kicker of the page: the same caps label as a section's, with the + brand's dot. */ +.eyebrow { + display: inline-flex; + align-items: flex-start; + gap: var(--sp-2); + justify-self: start; + margin-bottom: var(--sp-5); + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + color: var(--accent); } -.card h3 { - margin-bottom: 8px; +.eyebrow .dot { + width: 8px; + height: 8px; + border-radius: 50%; + background: linear-gradient(135deg, var(--brand-aqua), var(--brand-teal)); + box-shadow: 0 0 0 4px color-mix(in srgb, var(--accent) 16%, transparent); + flex: none; + /* Centred on the first line, however many the label takes: half its + line less half the dot. */ + margin-top: calc(0.8em - 4px); } -.card p { - color: var(--ink-soft); - font-size: 0.97rem; +/* One sentence per line: the second, in the gradient, starts its own. The + gradient belongs to this line alone. */ +.hero h1 { + font-size: var(--fs-hero); + font-weight: var(--fw-display); + letter-spacing: var(--tr-hero); + line-height: 1.02; + text-wrap: balance; + margin-bottom: var(--sp-7); } -.note { - font-size: 0.9rem; - color: var(--muted); - margin-top: 12px; +.hero h1 .grad { + display: block; + width: fit-content; + padding-bottom: 0.06em; } -/* โ”€โ”€ Start here: three steps โ”€โ”€ */ +/* On a laptop's short screen the statement steps down, so the drawing's + three rows still come into the first screen. */ +@media (min-width: 941px) and (max-height: 860px) { + .hero { + padding-top: var(--sp-9); + } -.steps { - list-style: none; - margin: 0; - padding: 0; - display: grid; - gap: 0; + .hero h1 { + font-size: min(var(--fs-hero), 7.4vh); + } } -.steps > li { - position: relative; - display: grid; - grid-template-columns: 44px minmax(0, 1fr); - gap: 0 18px; - padding-bottom: 40px; +.grad { + background: linear-gradient(100deg, var(--grad-a) 0%, var(--grad-b) 55%, var(--grad-c) 100%); + -webkit-background-clip: text; + background-clip: text; + color: transparent; } -.steps > li:last-child { - padding-bottom: 0; +.lead { + font-size: var(--fs-1); + line-height: var(--lh-read); + color: var(--ink-2); + max-width: 36em; } -/* The line that joins the numbers. */ -.steps > li::before { - content: ""; - position: absolute; - left: 21px; - top: 46px; - bottom: 6px; - width: 2px; - background: linear-gradient(var(--line-strong), var(--line)); +.actions { + display: flex; + flex-wrap: wrap; + gap: var(--sp-3); + margin-top: var(--sp-7); } -.steps > li:last-child::before { - display: none; +.hero-side .actions { + margin-top: 0; } -.step-num { - width: 44px; - height: 44px; - border-radius: 50%; +/* What goes in and what comes out, as a spec rather than a sentence. */ +.spec { display: grid; - place-items: center; - font-weight: 800; - font-size: 1.05rem; - color: var(--on-accent); - background: linear-gradient(135deg, var(--brand-aqua), var(--brand-teal)); + gap: var(--sp-2); + margin: 0; + font-size: var(--fs-n1); + line-height: var(--lh-ui); + max-width: 34em; } -:root[data-theme="light"] .step-num { - background: linear-gradient(135deg, #0b7a68, #05594c); +.spec div { + display: grid; + grid-template-columns: 3.6rem minmax(0, 1fr); + gap: var(--sp-3); + align-items: baseline; } -.step-body h3 { - font-size: 1.25rem; - margin: 8px 0 6px; +.spec dt { + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + color: var(--ink-3); } -.step-body > p { - color: var(--ink-soft); - margin-bottom: 14px; - max-width: 46em; +.spec dd { + margin: 0; + color: var(--ink-2); } -.step-body > .code + .code, -.step-body > .code + .out { - margin-top: 10px; -} +/* On a phone the drawing comes into the first screen: a shorter lead, no + second GitHub button (the header has one), and the spec after the + drawing rather than before it. */ +@media (max-width: 600px) { + .hero { + padding: var(--sp-7) 0 var(--sp-9); + } -.step-body .cols-2 { - align-items: start; -} + .hero h1 { + margin-bottom: var(--sp-4); + } -@media (max-width: 600px) { - .steps > li { - grid-template-columns: 34px minmax(0, 1fr); - gap: 0 12px; + .eyebrow { + margin-bottom: var(--sp-4); } - .step-num { - width: 34px; - height: 34px; - font-size: 0.95rem; + .hero .hero-side .actions { + margin-top: var(--sp-5); } - .steps > li::before { - left: 16px; - top: 36px; + .hero .demo { + margin-top: var(--sp-6); } -} -/* โ”€โ”€ Code โ”€โ”€ */ + .hero .spec { + margin-top: var(--sp-6); + } -.code { - position: relative; - background: var(--code-bg); - color: var(--code-ink); - border-radius: var(--radius-sm); - border: 1px solid var(--code-border); -} - -.code pre { - margin: 0; - padding: 16px 64px 16px 18px; - overflow-x: auto; - line-height: 1.6; - font-size: 0.86rem; - tab-size: 8; -} + .lead { + font-size: var(--fs-0); + } -/* On a phone every block wraps rather than run under the copy button; what - is copied is the text, not the wrapping. */ -@media (max-width: 600px) { - .code pre { - white-space: pre-wrap; - overflow-wrap: anywhere; + .hero .btn-gh { + display: none; } } -/* A one-line command that would hide under the copy button wraps instead; - what is copied is the text, not the wrapping. */ -.code.one-line pre { - white-space: pre-wrap; - overflow-wrap: anywhere; -} +/* โ”€โ”€ Across the whole genome: the run's four numbers โ”€โ”€ */ -.code .p { - color: var(--code-muted); - user-select: none; +/* The counts of the whole CDC1551 run, set as figures and coloured by what + they count, as the drawings colour it: Bakta's amber for the calls that + disagree, grey for what is dropped, the reference's violet for the genes + put back. A label in the margin column says they are the genome's, not + the 12.4 kb of Figure 1. */ +.numbers { + padding: 0 0 var(--sp-11); } -.code .c { - color: var(--code-muted); +.numbers .wrap { + display: grid; + grid-template-columns: var(--col-margin) minmax(0, 1fr); + column-gap: var(--col-gap); + align-items: start; } -.code .f { - color: var(--code-flag); +.numbers-head { + display: grid; + gap: var(--sp-1); + padding-top: var(--sp-2); + font-size: var(--fs-n1); + line-height: var(--lh-ui); + color: var(--ink-2); +} + +.numbers-head .numbers-label { + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + line-height: var(--lh-label); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + color: var(--ink-3); + margin-bottom: var(--sp-1); } -.code .v { - color: var(--code-value); +.numbers-head a { + justify-self: start; + margin-top: var(--sp-2); + font-weight: var(--fw-semibold); } -.copy { - position: absolute; - top: 10px; - right: 10px; - border: 1px solid var(--copy-border); - background: var(--copy-bg); - color: var(--code-ink); - font: 600 0.78rem var(--font); - padding: 6px 10px; - border-radius: 7px; - cursor: pointer; +.numbers-list { + list-style: none; + margin: 0; + padding: 0; + display: grid; + grid-template-columns: repeat(4, minmax(0, 1fr)); + font-size: var(--fs-n1); + line-height: var(--lh-ui); + color: var(--ink-3); } -.copy:hover { - background: var(--copy-hover); +.numbers-list li { + padding: 0 var(--sp-6); + border-left: 1px solid var(--line-1); + text-wrap: balance; } -.copy.done { - color: var(--copy-done); - border-color: currentColor; +.numbers-list li:first-child { + padding-left: 0; + border-left: 0; } -.code-title { - font: 600 0.75rem var(--font); - text-transform: uppercase; - letter-spacing: 0.06em; - color: var(--code-muted); - padding: 10px 18px 0; +.numbers-list b { + display: block; + margin-bottom: var(--sp-2); + font-size: var(--fs-10); + font-weight: var(--fw-light); + line-height: 1; + letter-spacing: var(--tr-numeral); + font-variant-numeric: tabular-nums; + color: var(--ink-1); } -.code-title + pre { - padding-top: 8px; +.numbers-list .n-settled b { + color: var(--bakta); } -kbd { - display: inline-block; - min-width: 1.6em; - padding: 0.05em 0.45em; - border: 1px solid var(--line-strong); - border-bottom-width: 2px; - border-radius: 6px; - background: var(--bg-subtle); - font: 600 0.8em var(--mono); - text-align: center; - color: var(--ink); +.numbers-list .n-dropped b { + color: var(--ink-3); } -.out { - margin-top: 10px; - background: var(--bg-subtle); - border: 1px dashed var(--line-strong); - border-radius: var(--radius-sm); - padding: 12px 16px; - font-family: var(--mono); - font-size: 0.84rem; - line-height: 1.55; - color: var(--ink-soft); - white-space: pre; - overflow-x: auto; +.numbers-list .n-back b { + color: var(--ref); } -.out .ok { - color: var(--ok); - font-weight: 700; +@media (max-width: 1100px) { + .numbers-list b { + font-size: var(--fs-7); + } } -.out .cm { - color: var(--muted); -} +@media (max-width: 940px) { + .numbers .wrap { + grid-template-columns: minmax(0, 1fr); + row-gap: var(--sp-6); + } -/* โ”€โ”€ Tabs โ”€โ”€ */ + .numbers-head { + grid-template-columns: auto auto; + justify-content: start; + align-items: baseline; + column-gap: var(--sp-4); + padding-top: 0; + } -.tabs { - display: flex; - gap: 4px; - margin-bottom: 10px; - flex-wrap: wrap; -} + .numbers-head .numbers-label { + grid-column: 1 / -1; + } -.tab { - border: 1px solid var(--line); - background: var(--bg-elev); - color: var(--ink-soft); - font: 600 0.86rem var(--font); - padding: 6px 12px; - border-radius: 8px; - cursor: pointer; + .numbers-head a { + margin-top: 0; + } } -.tab[aria-selected="true"] { - background: var(--ink); - border-color: var(--ink); - color: var(--bg); -} +@media (max-width: 600px) { + .numbers { + padding-bottom: var(--sp-10); + } -.tabpanel[hidden] { - display: none; -} + .numbers-head { + grid-template-columns: minmax(0, 1fr); + } -/* โ”€โ”€ Command builder โ”€โ”€ */ + .numbers-list { + grid-template-columns: repeat(2, minmax(0, 1fr)); + row-gap: var(--sp-6); + } -.builder { - display: grid; - grid-template-columns: minmax(0, 0.85fr) minmax(0, 1.15fr); - gap: 24px; - align-items: start; - margin-top: 22px; -} + .numbers-list li:nth-child(3) { + padding-left: 0; + border-left: 0; + } -@media (max-width: 900px) { - .builder { - grid-template-columns: minmax(0, 1fr); + .numbers-list b { + font-size: var(--fs-6); } } -.q { - margin-bottom: 18px; -} +/* โ”€โ”€ Gene tracks: the drawings โ”€โ”€ */ -.q-title { - font-weight: 700; - margin-bottom: 8px; - font-size: 0.95rem; +/* Every drawing is an SVG of rows on one coordinate axis. The colours are + classes, so a drawing follows the theme like the rest of the page. */ +.trk { + display: block; + width: 100%; + height: auto; + overflow: visible; + font-family: var(--font); } -.seg { - display: flex; - flex-wrap: wrap; - gap: 6px; +.trk text { + fill: var(--ink-soft); + font-size: 12px; } -.seg button { - border: 1px solid var(--line-strong); - background: var(--bg-elev); - color: var(--ink); - font: 600 0.88rem var(--font); - padding: 7px 12px; - border-radius: 9px; - cursor: pointer; +.trk .row-label { + fill: var(--ink-2); + font-size: 11px; + font-weight: var(--fw-semibold); + letter-spacing: 0.02em; } -.seg button[aria-pressed="true"] { - background: var(--accent); - border-color: var(--accent); - color: var(--on-accent); +.trk .row-line { + stroke: var(--line-strong); + stroke-width: 1; } -.opt { - display: grid; - grid-template-columns: auto 1fr; - gap: 2px 10px; - align-items: start; - padding: 10px 12px; - border: 1px solid var(--line); - border-radius: 10px; - background: var(--bg-elev); - margin-bottom: 8px; - cursor: pointer; +/* The dots that lead a row's name to its row. */ +.trk .row-lead { + stroke: var(--line-3); + stroke-width: 1.2; + stroke-linecap: round; + stroke-dasharray: 0 3.5; } -.opt:hover { - border-color: var(--line-strong); +.trk .row-frame { + transition: opacity var(--dur-reveal) var(--ease); } -.opt input[type="checkbox"] { - width: 18px; - height: 18px; - margin: 2px 0 0; - accent-color: var(--accent); +.trk .axis line, +.trk .axis path { + stroke: var(--line-strong); } -.opt b { - font-weight: 650; - font-size: 0.93rem; +.trk .axis text { + fill: var(--muted); + font-size: 10px; + font-family: var(--mono); } -.opt small { - grid-column: 2; - color: var(--muted); - font-size: 0.84rem; - line-height: 1.4; +.trk .g { + stroke-width: 1.4; + transition: opacity var(--dur-reveal) var(--ease), transform var(--dur-reveal) var(--ease); } -.explain { - margin: 12px 0 0; - padding: 0; - list-style: none; - display: grid; - gap: 6px; - font-size: 0.87rem; +.trk .g-bakta { + fill: color-mix(in srgb, var(--bakta) 26%, transparent); + stroke: var(--bakta); } -.explain li { - display: grid; - grid-template-columns: minmax(150px, max-content) 1fr; - gap: 12px; - color: var(--ink-soft); +.trk .g-ref { + fill: color-mix(in srgb, var(--ref) 26%, transparent); + stroke: var(--ref); } -.explain code { - white-space: nowrap; - justify-self: start; +.trk .g-amap { + fill: var(--amap); + stroke: var(--amap); } -@media (max-width: 560px) { - .explain li { - grid-template-columns: minmax(0, 1fr); - gap: 2px; - } +.trk .g-drop { + fill: transparent; + stroke: var(--drop); + stroke-dasharray: 4 3; } -/* โ”€โ”€ What AMAP decides: one card per case โ”€โ”€ */ +.trk .g-pseudo { + fill: url(#hatch); + stroke: var(--pseudo); +} -.cases { - display: grid; - grid-template-columns: repeat(2, minmax(0, 1fr)); - gap: 18px; +.trk .hatch-line { + stroke: var(--pseudo); + stroke-width: 2; } -@media (max-width: 860px) { - .cases { - grid-template-columns: minmax(0, 1fr); - } +.trk .gname { + font-size: 11.5px; + font-weight: var(--fw-semibold); + fill: var(--ink); } -.case { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; - padding: 18px 18px 16px; - display: flex; - flex-direction: column; - gap: 10px; - min-width: 0; +.trk .gname-on { + fill: var(--on-track); } -.case-wide { - grid-column: 1 / -1; +/* A name on a hatched pseudogene keeps a ring of the plate round each + letter, so the hatching never runs through it. */ +.trk .gname-hatch { + paint-order: stroke; + stroke: var(--halo); + stroke-width: 3px; + stroke-linejoin: round; } -.case-head { - display: flex; - align-items: baseline; - gap: 10px; - flex-wrap: wrap; +.trk .gname-above { + font-size: 10.5px; + paint-order: stroke; + stroke: var(--halo); + stroke-width: 3px; + stroke-linejoin: round; } -.case-tag { +.trk .gsub { + font-size: 10px; + fill: var(--muted); +} + +.trk .gap-mark { + stroke: var(--ref); + stroke-width: 2; +} + +/* What a figure points at: a hatched band with an edge of its own, in the + colour of what it marks, over the bands that join the rows. */ +.trk .diff { + stroke: var(--warn); + stroke-width: 1.5; +} + +.trk .diff-ref { + stroke: var(--ref); +} + +.trk .diff-hatch { + stroke: var(--warn); + stroke-width: 1.2; + opacity: 0.55; +} + +.trk .diff-hatch-ref { + stroke: var(--ref); +} + +.trk .diff-label { + fill: var(--warn); + font-size: 10.5px; + font-weight: var(--fw-bold); + paint-order: stroke; + stroke: var(--halo); + stroke-width: 3px; + stroke-linejoin: round; +} + +.trk .diff-label.diff-label-ref { + fill: var(--ref); +} + +.trk .g-cut { + stroke-dasharray: none; +} + +/* Where AMAP moved an end of Bakta's call. */ +.trk .moved { + fill: var(--warn); +} + +.trk .edge { + stroke: var(--ink-soft); + stroke-width: 2; +} + +.trk .edge-label { + fill: var(--muted); + font-size: 10px; +} + +/* The bands from a call to the protein placed on it: green and solid-edged + where the two agree, orange with dashed slanted edges where they do not. */ +.trk .rib { + stroke: none; +} + +.trk .rib-g { + transition: opacity var(--dur-reveal) var(--ease); +} + +.trk .rib-agree { + fill: color-mix(in srgb, var(--ok) 28%, transparent); +} + +.trk .rib-conflict { + fill: color-mix(in srgb, var(--warn) 36%, transparent); +} + +.trk .rib-edge { + fill: none; + stroke-width: 1; +} + +.trk .rib-edge-agree { + stroke: var(--ok); +} + +.trk .rib-edge-conflict { + stroke: var(--warn); + stroke-dasharray: 3 2; +} + +/* In the light theme the warning orange is the darker of its two, so the + edge of a marked stretch still stands out from the orange band under it. */ +:root[data-theme="light"] .trk .diff { + stroke: color-mix(in srgb, var(--warn) 78%, #000); +} + +:root[data-theme="light"] .trk .diff-ref { + stroke: color-mix(in srgb, var(--ref) 85%, #000); +} + +/* A call nothing aligns to, once the calls are compared: dashed. Once AMAP + has kept what it keeps, the ones it dropped turn grey and faint. */ +[data-phase="3"] .c1-call .g, +[data-phase="4"] .c1-call .g { + stroke-dasharray: 3 2; +} + +[data-phase="4"] .c1-call.ph-dim .g { + stroke: var(--drop); + fill: transparent; +} + +/* Every gene is a target of at least 14 by 24 px, and one stop of a drawing + the keyboard moves through with the arrows. */ +.trk .hit { + fill: transparent; + stroke: none; +} + +.trk .hot { + cursor: pointer; + transition: opacity var(--dur-reveal) var(--ease); +} + +.trk .hot.on .g { + stroke-width: 2.6; +} + +@media (hover: hover) { + .trk .hot:hover .g { + stroke-width: 2.6; + } +} + +.trk .hot:focus { + outline: none; +} + +.trk .hot:focus-visible .g { + stroke: var(--ink); + stroke-width: 3; + paint-order: stroke; +} + +.trk .hot:focus-visible .hit { + stroke: var(--ink); + stroke-width: 1.5; + stroke-dasharray: 3 2; +} + +/* A dropped call with the keyboard on it shows whole: solid, full strength. */ +.trk .hot.ph-dim:focus-visible { + opacity: 1; +} + +.trk .hot:focus-visible .g { + stroke-dasharray: none; +} + +/* Pointing at a count of the hero lights what it counts. */ +.trk[data-light] .hot, +.trk[data-light] .rib-g { + opacity: 0.2; +} + +.trk[data-light="agree"] [data-k="agree"], +.trk[data-light="fixed"] [data-k="fixed"], +.trk[data-light="drop"] [data-k="drop"] { + opacity: 1; +} + +/* The dropped calls overlap: lit, each is a whole outline, not dashes. */ +.trk[data-light="drop"] [data-k="drop"] .g { + stroke-dasharray: none; +} + +/* Elements a step has just brought in, outlined for a moment. */ +@keyframes just-on { + from { + stroke: var(--accent); + stroke-width: 3.5; + } +} + +.trk .just-on .g, +.trk .just-on .diff, +.trk .just-on .edge, +.trk .just-on.rib-g .rib-edge { + animation: just-on 0.7s ease-out; +} + +/* The tip: over the drawing, its caret on the gene, where there is a mouse + and room; a line of text under the drawing everywhere else. */ +.demo-tip { + position: absolute; + z-index: 5; + max-width: 290px; + padding: var(--sp-2) var(--sp-3); + border-radius: var(--r-3); + background: var(--overlay); + border: 1px solid var(--line-2); + box-shadow: var(--shadow-3); + font-size: var(--fs-n2); + line-height: var(--lh-ui); + color: var(--ink-2); +} + +.demo-tip::after { + content: ""; + position: absolute; + left: var(--caret, 50%); + bottom: -6px; + width: 10px; + height: 10px; + margin-left: -5px; + background: var(--overlay); + border: solid var(--line-2); + border-width: 0 1px 1px 0; + transform: rotate(45deg); +} + +.demo-tip.below::after { + bottom: auto; + top: -6px; + transform: rotate(225deg); +} + +.demo-tip b { + color: var(--ink); +} + +.demo-tip code { + font-size: var(--fs-n2); +} + +.fig-readout { + display: none; +} + +.tips-inline > .demo-tip { + display: none; +} + +.tips-inline > .fig-readout { + display: block; + min-height: calc(3 * 1.4em + 14px); + margin: 8px 0 0; + padding: var(--sp-2) var(--sp-3); + border-radius: var(--r-3); + background: var(--bg-subtle); + font-size: var(--fs-n2); + line-height: 1.4; + color: var(--ink-soft); +} + +.fig-readout b { + color: var(--ink); +} + +.fig-readout .say-hint { + color: var(--muted); +} + +/* "Point at" where there is a pointer, "Tap" on a touch screen. */ +.say-touch { + display: none; +} + +@media (hover: none) { + .say-hover { + display: none; + } + + .say-touch { + display: inline; + } +} + +/* โ”€โ”€ The animated stretch of genome in the hero โ”€โ”€ */ + +/* A plate: the surface every drawing stands on (level 2: its own fill, an + edge and a shadow, 16px corners, 24px in). The hero's holds Figure 1: a + head of its label, its four phases and Play; the drawing, its counts read + out in a column at its right; and under both, its caption and key. */ +.demo { + background: var(--surface-2); + border: 1px solid var(--line-2); + border-radius: var(--r-5); + padding: var(--sp-5) var(--plate-pad); + box-shadow: var(--shadow-2); + margin: 0; + display: grid; + grid-template-columns: minmax(0, 1fr) 13.5rem; + grid-template-areas: + "head head" + "stage tally" + "cap cap" + "key key"; + column-gap: var(--sp-7); + align-items: center; +} + +.demo-head { + grid-area: head; + display: grid; + grid-template-columns: auto minmax(0, 1fr) auto; + grid-template-areas: "label step ctrl"; + align-items: center; + gap: var(--sp-2) var(--sp-4); + margin: 0 0 var(--sp-4); + padding-bottom: var(--sp-3); + border-bottom: 1px solid var(--line-1); +} + +.demo-head .fig-n { + grid-area: label; +} + +.demo-stage { + grid-area: stage; + position: relative; + min-width: 0; +} + +.demo-ctrl { + grid-area: ctrl; + display: inline-flex; + align-items: center; + gap: 10px; +} + +.demo-ctrl .mini-btn { + display: inline-grid; + justify-items: center; + align-content: center; + min-width: 4.4em; +} + +/* Play and Pause share one cell, so the button is as wide as Pause even + while it says Play, and the caption beside it never rewraps. */ +.demo-ctrl .mini-btn > span { + grid-area: 1 / 1; +} + +.demo-ctrl .mini-btn > .say-2, +.demo-ctrl .mini-btn.said-2 > .say-1 { + visibility: hidden; +} + +.demo-ctrl .mini-btn.said-2 > .say-2 { + visibility: visible; +} + +/* Four segments, one per phase: each fills while its phase plays, and takes + the drawing to its phase when pressed. The words may wrap, in room kept + for them. */ +.demo-step { + grid-area: step; + display: flex; + flex-wrap: wrap; + align-items: center; + gap: 2px 12px; + min-height: 1.6em; + color: var(--ink-1); + font-weight: var(--fw-semibold); +} + +.hero-segs { + display: inline-flex; + gap: 4px; + flex: none; +} + +.hero-seg { + position: relative; + width: 30px; + height: 24px; + padding: 0; + border: 0; + border-radius: var(--r-1); + background: none; + cursor: pointer; +} + +.hero-seg::before, +.hero-seg i { + content: ""; + position: absolute; + left: 0; + right: 0; + top: 10px; + height: 4px; + border-radius: 2px; +} + +/* A phase still to come is drawn at 3:1, so the control shows. */ +.hero-seg::before { + background: var(--line-3); +} + +@media (hover: hover) { + .hero-seg:hover::before { + background: var(--ink-3); + } +} + +@media (hover: hover) { + .hero-seg:hover i { + background: var(--accent-hover); + } +} + +.hero-seg i { + background: var(--accent); + transform: scaleX(0); + transform-origin: left; +} + +.hero-seg.done i, +.demo:not(.running) .hero-seg.now i { + transform: scaleX(1); +} + +.demo.running .hero-seg.now i { + animation: hero-fill var(--dwell, 2s) linear forwards; +} + +.demo.paused .hero-seg.now i { + animation-play-state: paused; +} + +@keyframes hero-fill { + from { + transform: scaleX(0); + } + + to { + transform: scaleX(1); + } +} + +.hero-phase { + flex: 1 1 12em; + min-width: 0; +} + +/* The caption under the drawing and its counts: what the figure shows, in a + bold first sentence, then the key to its marks. */ +.demo-cap { + grid-area: cap; + display: grid; + gap: var(--sp-3); + margin-top: var(--sp-4); + padding-top: var(--sp-4); + border-top: 1px solid var(--line-1); + font-size: var(--fs-n2); + line-height: var(--lh-ui); + color: var(--ink-3); +} + +.demo-cap p { + max-width: 56em; + text-wrap: pretty; +} + +.demo-cap b { + color: var(--ink-1); + font-weight: var(--fw-semibold); +} + +/* The key closes the caption on a wide screen; narrower, it comes straight + under the drawing, before the counts, so the colours meet their names. */ +.demo > .legend { + grid-area: key; + margin-top: var(--sp-3); +} + +/* Narrower, the counts go under the drawing, three in a row, so the drawing + keeps the width at which its genes are named. */ +@media (max-width: 1100px) { + .demo { + grid-template-columns: minmax(0, 1fr); + grid-template-areas: "head" "stage" "key" "tally" "cap"; + } +} + +@media (max-width: 600px) { + .demo { + padding: var(--sp-4) var(--sp-4) var(--sp-4); + } + + /* The label and Play on one line, the phases under them. */ + .demo-head { + grid-template-columns: minmax(0, 1fr) auto; + grid-template-areas: "label ctrl" "step step"; + } + + .demo-step { + min-height: calc(24px + 2px + 1.55em); + font-size: var(--fs-n1); + } +} + +.legend { + display: flex; + flex-wrap: wrap; + gap: var(--sp-2) var(--sp-5); + margin-top: var(--sp-3); + font-size: var(--fs-n2); + line-height: var(--lh-ui); + color: var(--ink-3); +} + +.legend span { + display: inline-flex; + align-items: center; + gap: 6px; +} + +/* The chips use the very fills of the drawing they explain. */ +.chip { + width: 18px; + height: 10px; + border-radius: 3px; + display: inline-block; + border: 1.5px solid; + flex: none; +} + +.chip.bakta { + border-color: var(--bakta); + background: color-mix(in srgb, var(--bakta) 26%, transparent); +} + +.chip.ref { + border-color: var(--ref); + background: color-mix(in srgb, var(--ref) 26%, transparent); +} + +.chip.ref.short { + border-right: 0; + border-radius: 3px 0 0 3px; + -webkit-mask-image: linear-gradient(90deg, #000 45%, rgba(0, 0, 0, 0.15)); + mask-image: linear-gradient(90deg, #000 45%, rgba(0, 0, 0, 0.15)); +} + +.chip.amap { + border-color: var(--amap); + background: var(--amap); +} + +.chip.amap.partial { + border-radius: 3px 0 0 3px; + clip-path: polygon(0 0, 100% 0, 78% 25%, 100% 50%, 78% 75%, 100% 100%, 0 100%); +} + +.chip.c1 { + border-color: var(--bakta); + border-style: dashed; + background: color-mix(in srgb, var(--bakta) 26%, transparent); +} + +.chip.drop { + border-color: var(--drop); + border-style: dashed; + background: transparent; +} + +.chip.band-agree { + border: solid var(--ok); + border-width: 0 1px; + border-radius: 0; + background: color-mix(in srgb, var(--ok) 28%, transparent); +} + +.chip.band-conflict { + border: dashed var(--warn); + border-width: 0 1px; + border-radius: 0; + background: color-mix(in srgb, var(--warn) 36%, transparent); +} + +.chip.pseudo { + border-color: var(--pseudo); + background: repeating-linear-gradient(135deg, var(--pseudo) 0 2px, transparent 2px 5px); +} + +.chip.diff { + border-color: var(--warn); + background: repeating-linear-gradient(135deg, color-mix(in srgb, var(--warn) 55%, transparent) 0 1.2px, transparent 1.2px 3.6px); +} + +.chip.diff-ref { + border-color: var(--ref); + background: repeating-linear-gradient(135deg, color-mix(in srgb, var(--ref) 55%, transparent) 0 1.2px, transparent 1.2px 3.6px); +} + +/* The same darker edges as the marks they name, in the light theme. */ +:root[data-theme="light"] .chip.diff { + border-color: color-mix(in srgb, var(--warn) 78%, #000); +} + +:root[data-theme="light"] .chip.diff-ref { + border-color: color-mix(in srgb, var(--ref) 85%, #000); +} + +.chip.moved { + width: 9px; + height: 6px; + border: 0; + border-radius: 0; + background: var(--warn); + clip-path: polygon(0 0, 100% 0, 50% 100%); +} + +/* The counts of Figure 1, read out in a column at the drawing's right: each + a figure in the colour of what it counts (the bands' green and amber, the + grey of a dropped call), beside what it counts. */ +.tally { + grid-area: tally; + display: grid; + gap: var(--sp-1); + align-self: stretch; + align-content: center; + padding-left: var(--sp-6); + border-left: 1px solid var(--line-1); + font-size: var(--fs-n2); + line-height: var(--lh-ui); + color: var(--ink-3); +} + +.tally > div { + display: grid; + grid-template-columns: 1.6ch minmax(0, 1fr); + align-items: center; + column-gap: var(--sp-4); + margin: 0 calc(-1 * var(--sp-2)); + padding: var(--sp-2); + border-radius: var(--r-3); + text-wrap: balance; + transition: background-color var(--dur-state) var(--ease); +} + +.tally > .lit { + background: var(--surface-3); + color: var(--ink-2); +} + +.tally > [role="button"] { + cursor: pointer; +} + +/* A count pressed to stay lit keeps a frame. */ +.tally > [aria-pressed="true"] { + box-shadow: inset 0 0 0 1px var(--line-3); +} + +.tally b { + font-size: var(--fs-6); + font-weight: var(--fw-light); + letter-spacing: var(--tr-numeral); + line-height: 1; + color: var(--ink-1); + font-variant-numeric: tabular-nums; + text-align: center; + transition: color var(--dur-reveal) var(--ease); +} + +.tally .t-agree b { + color: var(--ok); +} + +.tally .t-fixed b { + color: var(--warn); +} + +.tally .t-drop b { + color: var(--ink-3); +} + +/* Before the calls are compared, the counts wait in grey, still readable. */ +.tally .t-off b { + color: var(--ink-3); +} + +/* Under the drawing, three in a row, each figure over what it counts. */ +@media (max-width: 1100px) { + .tally { + grid-template-columns: repeat(3, minmax(0, 1fr)); + gap: var(--sp-3); + margin-top: var(--sp-4); + padding: var(--sp-4) 0 0; + border-left: 0; + border-top: 1px solid var(--line-1); + } + + .tally > div { + grid-template-columns: minmax(0, 1fr); + align-items: start; + row-gap: var(--sp-1); + } + + .tally b { + font-size: var(--fs-4); + text-align: left; + } +} + +/* On the narrowest phones a column of a third is too thin for its words: + each count takes a row, its figure before what it counts. */ +@media (max-width: 420px) { + .tally { + grid-template-columns: minmax(0, 1fr); + row-gap: var(--sp-1); + } + + .tally > div { + display: flex; + gap: var(--sp-3); + align-items: baseline; + } +} + +/* The phases: each class reveals one more layer. A hidden layer leaves the + pointer and the keyboard too: it fades out, then turns invisible. */ +[data-phase] .ph-ref, +[data-phase] .ph-case, +[data-phase] .ph-amap { + opacity: 0; + visibility: hidden; + transition: opacity var(--dur-reveal) var(--ease), visibility 0s linear var(--dur-reveal); +} + +[data-phase="2"] .ph-ref, +[data-phase="3"] .ph-ref, +[data-phase="4"] .ph-ref, +[data-phase="3"] .ph-case, +[data-phase="4"] .ph-case, +[data-phase="4"] .ph-amap { + opacity: 1; + visibility: visible; + transition-delay: 0s; +} + +/* The rows still to come show their names a step quieter (still 4.5:1 and + more) and their lines faintly, so the reader sees where they will come. */ +[data-phase="1"] :is(.fr-ref, .fr-amap) .row-label, +[data-phase="2"] .fr-amap .row-label, +[data-phase="3"] .fr-amap .row-label { + fill: var(--ink-3); + font-weight: var(--fw-medium); +} + +[data-phase="1"] :is(.fr-ref, .fr-amap) :is(.row-line, .row-lead), +[data-phase="2"] .fr-amap :is(.row-line, .row-lead), +[data-phase="3"] .fr-amap :is(.row-line, .row-lead) { + opacity: 0.35; +} + +[data-phase="4"] .ph-dim { + opacity: 0.55; +} + +/* โ”€โ”€ Sections โ”€โ”€ */ + +/* A section is a chapter: 96px above and below on a wide screen, a hairline + between chapters. Its head hangs its number and label in the margin + column, and puts the title and lead in the main one; steps, the FAQ and + running text follow the main column, and figures take the whole width. */ +.section { + padding: var(--sp-12) 0; + border-top: 1px solid var(--line-1); +} + +@media (max-width: 940px) { + .section { + padding: var(--sp-11) 0; + } +} + +@media (max-width: 600px) { + .section { + padding: var(--sp-10) 0; + } +} + +/* Two widths on the front page: running text ends at one reading measure, and + everything else (cards, drawings, the benchmark, the report) spans the + whole column. */ +.section { + --measure: 42rem; +} + +main { + counter-reset: chapter figure; +} + +.section-head { + margin-bottom: var(--sp-9); +} + +.section-head h2 { + max-width: 20em; +} + +/* The kicker: the chapter's number, drawn large in the accent (not read), + over its label in caps. */ +.kicker { + display: flex; + align-items: center; + gap: var(--sp-3); + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + line-height: var(--lh-label); + text-transform: uppercase; + letter-spacing: var(--tr-caps); + color: var(--accent); + margin-bottom: var(--sp-4); +} + +.section-head .kicker { + color: var(--ink-3); +} + +.section-head .kicker::before { + counter-increment: chapter; + content: counter(chapter, decimal-leading-zero); + content: counter(chapter, decimal-leading-zero) / ""; + font-size: var(--fs-10); + font-weight: var(--fw-light); + line-height: 0.82; + letter-spacing: var(--tr-numeral); + font-variant-numeric: tabular-nums; + text-transform: none; + color: var(--accent); +} + +/* Not the kicker: a paragraph of the head would otherwise outrank it and turn + it grey and full size. */ +.section-head p:not(.kicker, .pull) { + color: var(--ink-2); + font-size: var(--fs-1); + line-height: var(--lh-read); + margin-top: var(--sp-4); + max-width: var(--measure); +} + +/* A figure the chapter stands on, pulled out into the margin: the number + large, what it counts under it. */ +.pull { + font-size: var(--fs-n1); + line-height: var(--lh-ui); + color: var(--ink-2); +} + +.pull b { + display: block; + margin-bottom: var(--sp-2); + font-size: var(--fs-10); + font-weight: var(--fw-light); + line-height: 1; + letter-spacing: var(--tr-numeral); + font-variant-numeric: tabular-nums; + color: var(--accent); +} + +/* On a wide screen: the margin column, then the main one. The number's top + meets the title's capitals. */ +@media (min-width: 941px) { + .section-head { + display: grid; + grid-template-columns: var(--col-margin) minmax(0, 1fr); + grid-template-rows: auto auto 1fr; + column-gap: var(--col-gap); + align-items: start; + } + + .section-head > * { + grid-column: 2; + } + + .section-head > .kicker { + grid-column: 1; + grid-row: 1 / span 2; + flex-direction: column; + align-items: flex-start; + gap: var(--sp-4); + margin: 0.32rem 0 0; + } + + /* A head with a figure pulled out keeps its number level with the title, + and the figure level with the lead. */ + .section-head:has(> .pull) > .kicker { + grid-row: 1; + } + + .section-head > .pull { + grid-column: 1; + grid-row: 2 / span 2; + margin-top: var(--sp-6); + padding-top: var(--sp-5); + border-top: 1px solid var(--line-2); + } + + /* Steps, questions and the notes between figures read in the main column. */ + .section .wrap > :is(.steps, .faq) { + margin-left: calc(var(--col-margin) + var(--col-gap)); + } +} + +/* A head of number and title alone (the questions) keeps its list close. */ +@media (min-width: 941px) { + .section-head:not(:has(> p:not(.kicker))) { + margin-bottom: var(--sp-6); + } +} + +@media (max-width: 940px) { + .section-head { + margin-bottom: var(--sp-7); + } + + .section-head .kicker::before { + font-size: var(--fs-6); + margin-right: var(--sp-1); + } + + .pull { + margin-top: var(--sp-6); + padding-left: var(--sp-4); + border-left: 2px solid var(--accent-line); + max-width: var(--measure); + } + + .pull b { + font-size: var(--fs-6); + } +} + +@media (max-width: 600px) { + .section-head { + margin-bottom: var(--sp-6); + } + + .section-head p:not(.kicker, .pull) { + font-size: var(--fs-0); + line-height: var(--lh-body); + } +} + +/* A figure's label and title, over it: "Figure n" in caps, counted down the + page, then what it shows. */ +.fig-title { + display: flex; + flex-wrap: wrap; + align-items: baseline; + gap: var(--sp-1) var(--sp-3); + margin: 0 0 var(--sp-4); + font-size: var(--fs-n1); + font-weight: var(--fw-semibold); + line-height: var(--lh-ui); + color: var(--ink-1); +} + +.fig-n { + flex: none; + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + line-height: var(--lh-label); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + color: var(--accent); +} + +.fig-n::before { + counter-increment: figure; + content: "Figure " counter(figure); + content: "Figure " counter(figure) / ""; +} + +/* โ”€โ”€ The front page's rail of chapters โ”€โ”€ + Where there is room beside the column, the page's figure and chapters + stand in the margin as a ruler: a tick for each, numbered as the page + numbers them, on rows of two grid squares. The one being read takes the + accent on the rule and its name in full ink (site.js, as the docs' + contents). The room is 6.5rem of rail, 20px from the column and 12px or + more from the screen's edge: from 90em, which is 1440px with the usual + 16px text and wider when the reader sets larger text, so the names never + reach the column. Narrower, the rail goes: the header's search lists the + same chapters, and on a phone the bar under the header names the one in + view. */ +.rail { + display: none; +} + +@media (min-width: 90em) { + .rail { + --rail-w: 6.5rem; + display: block; + position: fixed; + z-index: 40; + top: 160px; + /* On a minor line of the graticule, 128px left of the column's edge; + where the margin is too narrow for that, 12px from the screen's. */ + left: max(var(--sp-3), calc(50% - 584px - 8 * var(--sp-4))); + width: var(--rail-w); + } +} + +.rail ol { + list-style: none; + margin: 0; + padding: 0; +} + +.rail a { + position: relative; + display: grid; + /* The number's column is sized in em, not ch, so a semibold current + chapter does not push its name along. */ + grid-template-columns: 1.3em minmax(0, 1fr); + column-gap: var(--sp-2); + align-items: center; + height: 32px; + padding-left: var(--sp-3); + border-left: 1px solid var(--line-2); + color: var(--ink-3); + font-size: var(--fs-n3); + line-height: var(--lh-label); + white-space: nowrap; + text-decoration: none; +} + +/* A tick on the rule for each, as the ruler under the header has. */ +.rail a::after { + content: ""; + position: absolute; + left: 0; + top: 50%; + width: 6px; + height: 1px; + background: var(--tick-major); +} + +/* A name may run 12px into the gutter (a semibold one does, by a pixel or + two), never further: past that it is cut short rather than run into the + column. */ +.rail a > span:last-child { + margin-right: calc(-1 * var(--sp-3)); + overflow: hidden; + text-overflow: ellipsis; +} + +.rail-n { + font-weight: var(--fw-bold); + letter-spacing: 0.04em; + font-variant-numeric: tabular-nums; + color: var(--accent); +} + +/* Figure 1 is named as the page names it, in caps in the accent. */ +.rail a.rail-fig { + display: flex; + font-weight: var(--fw-bold); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + color: var(--accent); +} + +@media (hover: hover) { + .rail a:hover { + color: var(--ink-1); + } +} + +@media (hover: hover) { + .rail a.rail-fig:hover { + color: var(--accent-hover); + } +} + +.rail a[aria-current]::before { + content: ""; + position: absolute; + left: -1px; + top: 7px; + bottom: 7px; + width: 2px; + border-radius: 1px; + background: var(--accent); +} + +.rail a[aria-current]::after { + width: 9px; + background: var(--accent); +} + +.rail a[aria-current]:not(.rail-fig) { + color: var(--ink-1); + font-weight: var(--fw-semibold); +} + +@media (pointer: coarse) { + .rail a { + height: 40px; + } +} + +.cols-2 { + display: grid; + grid-template-columns: repeat(2, minmax(0, 1fr)); + gap: var(--sp-5); +} + +@media (max-width: 820px) { + .cols-2 { + grid-template-columns: minmax(0, 1fr); + } +} + +/* A card: level 1, an edge on the page's own dark, 12px corners, 24px in. */ +.card { + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); + padding: var(--sp-6); +} + +.card h3 { + margin-bottom: var(--sp-2); +} + +.card p { + color: var(--ink-2); + font-size: var(--fs-n1); +} + +.note { + font-size: var(--fs-n1); + line-height: var(--lh-read); + color: var(--ink-3); + margin-top: var(--sp-3); +} + +/* A note under a figure, its caption: as wide as a line of text reads. */ +.section .wrap > .note { + max-width: 52rem; + margin-top: var(--sp-4); +} + +/* Words for the pointer the reader has: "point at" with a mouse, "tap" on a + touch screen. */ +.when-touch { + display: none; +} + +@media (hover: none) { + .when-hover { + display: none; + } + + .when-touch { + display: inline; + } +} + +/* โ”€โ”€ Start here: four steps โ”€โ”€ */ + +/* Each step under a hairline, its number hung in the margin column, large + and light, its title level with the number's top. */ +.steps { + list-style: none; + margin: 0; + padding: 0; + display: grid; + grid-template-columns: minmax(0, 1fr); + gap: 0; +} + +.steps > li { + position: relative; + padding: var(--sp-8) 0 var(--sp-10); + border-top: 1px solid var(--line-1); +} + +.steps > li:last-child { + padding-bottom: 0; +} + +.step-num { + display: block; + font-size: var(--fs-7); + font-weight: var(--fw-light); + line-height: 1; + letter-spacing: var(--tr-numeral); + font-variant-numeric: tabular-nums; + color: var(--accent); +} + +@media (min-width: 941px) { + /* The number hangs out of the main column, into the margin. */ + .step-num { + position: absolute; + top: calc(var(--sp-8) - 0.06em); + right: calc(100% + var(--col-gap)); + } +} + +.step-body h3 { + font-size: var(--fs-2); + margin: 0 0 var(--sp-3); +} + +.step-body > p { + color: var(--ink-2); + margin-bottom: var(--sp-4); + max-width: var(--measure, 42rem); +} + +.step-body > :last-child { + margin-bottom: 0; +} + +.step-body > .code + .code, +.step-body > .code + .out { + margin-top: var(--sp-3); +} + +.step-body .cols-2 { + align-items: start; +} + +/* What the end of a good run looks like, said before it is shown. */ +.step-body > .out-head { + margin: var(--sp-5) 0 0; + color: var(--ink-3); + font-size: var(--fs-n1); +} + +.step-body > .out-head + .out { + margin-top: var(--sp-2); +} + +/* The sample sheet, and what its columns take. */ +/* Split where the builder under it splits, so the file and the command + share a left column and the notes a right one. */ +.steps .sheet { + display: grid; + grid-template-columns: minmax(0, 0.85fr) minmax(0, 1.15fr); + gap: var(--sp-3) var(--sp-6); + align-items: center; +} + +.steps .sheet > .note { + margin: 0; +} + +/* Tab stops every six columns: each row's tab ends at the same column, wide + enough for the mark the Code section draws on it. Copy sits in the title + row, so the rows can run to the right edge and stay whole on a phone. */ +.steps .sheet .code pre { + tab-size: 6; + padding-right: 18px; +} + +@media (max-width: 900px) { + .steps .sheet { + grid-template-columns: minmax(0, 1fr); + } +} + +/* Narrower, the number sits beside the step's title, and the rest of the + step runs the whole width under both. */ +@media (max-width: 940px) { + .steps > li { + display: grid; + grid-template-columns: 2.6rem minmax(0, 1fr); + align-items: baseline; + padding: var(--sp-7) 0 var(--sp-9); + } + + .step-body { + display: contents; + } + + .step-body > * { + grid-column: 1 / -1; + } + + .step-body > h3 { + grid-column: 2; + } + + .step-num { + font-size: var(--fs-6); + } +} + +@media (max-width: 600px) { + .steps > li { + grid-template-columns: 2.2rem minmax(0, 1fr); + padding: var(--sp-6) 0 var(--sp-9); + } + + .step-num { + font-size: var(--fs-4); + } + + .step-body h3 { + font-size: var(--fs-1); + } + + /* What a run prints, smaller: it is there to be recognised, not read. */ + .step-body .out { + font-size: var(--fs-n3); + line-height: 1.45; + padding: var(--sp-2) var(--sp-3); + } +} + +/* โ”€โ”€ Code โ”€โ”€ */ + +.code { + position: relative; + background: var(--code-bg); + color: var(--code-ink); + border-radius: var(--r-4); + border: 1px solid var(--code-border); + /* The colour a block's sideways scroll shadow covers with, and the shadow. */ + --cover: var(--code-bg); + --shade: rgba(255, 255, 255, 0.18); +} + +:root[data-theme="light"] .code { + --shade: rgba(16, 27, 24, 0.2); +} + +.code pre { + margin: 0; + padding: var(--sp-4) var(--sp-10) var(--sp-4) var(--sp-5); + overflow-x: auto; + line-height: 1.7; + font-size: var(--fs-n2); + tab-size: 8; + /* The block's own corners, so its background does not square them off. */ + border-radius: calc(var(--r-4) - 1px); + /* A block wider than its box shades the edge it continues past. The covers + scroll with the text and the shadows stay put, so a shadow shows only + where there is more to see. */ + background: + linear-gradient(to right, var(--cover) 40%, transparent) left / 28px 100% no-repeat local, + linear-gradient(to left, var(--cover) 40%, transparent) right / 28px 100% no-repeat local, + radial-gradient(farthest-side at 0 50%, var(--shade), transparent) left / 12px 100% no-repeat scroll, + radial-gradient(farthest-side at 100% 50%, var(--shade), transparent) right / 12px 100% no-repeat scroll; +} + +/* On the front page a block wraps on a phone rather than run under the copy + button; the docs' blocks keep their layout (see below). A block with no + head keeps a strip above its text for Copy, instead of a margin down the + right of every line. */ +@media (max-width: 600px) { + .code pre { + white-space: pre-wrap; + } + + .code:not(.out):not(:has(> .code-title)) { + padding-top: 30px; + } + + .code:not(.out):not(:has(> .code-title)) > pre { + padding-top: var(--sp-2); + padding-right: var(--sp-4); + } + + .code:not(.out):not(:has(> .code-title)) > .copy { + top: 3px; + right: 6px; + padding: 3px 10px; + } +} + +/* A command wraps at the spaces between its words and never inside one, and + a line that wraps hangs the rest of itself four characters further in than + it starts (--i is the line's own indent). What is copied is the text, not + the wrapping. */ +.code.cmd pre, +.code.one-line pre { + white-space: pre-wrap; +} + +.code .ln { + display: block; + padding-left: calc((var(--i, 0) + 4) * 1ch); + text-indent: calc((var(--i, 0) + 4) * -1ch); +} + +.code :is(.t, .f, .v, .kv), +code.nw, +code .nw { + white-space: nowrap; +} + +/* A name with a hyphen in it, K-12, kept on one line (build_docs.py's tie). */ +.tie { + white-space: nowrap; +} + +/* Words a command line keeps together: its indent with its first word, and + its closing backslash with its last. pre keeps the indent's spaces. */ +.code .ln .nw { + white-space: pre; +} + +/* A path too long for a phone's line comes apart only after a /. */ +.code :is(.t, .f, .v).path { + white-space: inherit; +} + +.code .path .nw { + white-space: nowrap; +} + +/* A long piece of a chip with nothing to break at, a commit hash, breaks + anywhere rather than run off a phone's screen; a long word of a command + too, after any spaces it has. */ +code .brk, +.code :is(.t, .f, .v).brk, +.code .path .brk { + overflow-wrap: anywhere; +} + +.code :is(.t, .f, .v).brk { + white-space: inherit; +} + +/* Files, trees and program output keep their layout at every width, and + scroll. The commands a builder writes ([data-command], [data-check]) wrap + as on the front page. */ +.prose .code:not(.cmd) pre:not([data-command], [data-check]) { + white-space: pre; +} + +/* A block of one line has no columns to keep: it wraps, rather than hide + the end the text goes on to read (site.js marks it). */ +.prose .code.one-line:not(.cmd) pre:not([data-command], [data-check]) { + white-space: pre-wrap; + overflow-wrap: anywhere; +} + +.code .p { + color: var(--code-muted); + user-select: none; +} + +.code .c, +.code .tc { + color: var(--code-muted); +} + +/* A tree's comment goes under its file on a phone, from where the name + starts (--c), instead of off the right edge. */ +@media (max-width: 600px) { + .prose .code.tree .tc { + display: block; + white-space: normal; + padding-left: calc((var(--c, 0) + 2) * 1ch); + } +} + +.code .f { + color: var(--code-flag); +} + +.code .v { + color: var(--code-value); +} + +/* A tab of a tab-separated file, drawn as the sample sheet drawing draws it; + the character underneath is a real tab, which is what Copy takes. */ +.code .tsv-tab { + position: relative; +} + +.code .tsv-tab::before { + content: "\21E5"; + position: absolute; + left: 0.2em; + top: 50%; + transform: translateY(-50%); + padding: 0 0.35em; + border-radius: var(--r-1); + font-size: 0.85em; + line-height: 1.35; + color: var(--code-value); + background: color-mix(in srgb, var(--code-value) 14%, transparent); + pointer-events: none; +} + +.code.tsv pre { + tab-size: 6; +} + +.copy { + position: absolute; + top: var(--sp-2); + right: var(--sp-2); + border: 1px solid var(--copy-border); + background: var(--copy-bg); + color: var(--code-ink); + font: var(--fw-semibold) var(--fs-n2) var(--font); + min-height: 28px; + padding: 0 var(--sp-3); + border-radius: var(--r-2); + cursor: pointer; +} + +@media (hover: hover) { + .copy:hover { + background: var(--copy-hover); + border-color: var(--line-3); + } +} + +.copy.done { + color: var(--copy-done); + border-color: currentColor; +} + +/* "Copy" and "Copied" (or "Press Ctrl+C") in one cell, so the button keeps + its width when its word changes. */ +.copy { + display: inline-grid; + place-items: center; + align-content: center; +} + +.copy > span { + grid-area: 1 / 1; +} + +.copy > .say-2, +.copy:is(.done, .ask) > .say-1 { + visibility: hidden; +} + +.copy:is(.done, .ask) > .say-2 { + visibility: visible; +} + +/* A block's head: what it is (a terminal, a file, a command's output), with + Copy at its right. A file name is written as it is spelled. */ +.code-title { + display: flex; + align-items: center; + min-height: 40px; + padding: 0 128px 0 var(--sp-5); + border-bottom: 1px solid var(--code-border); + font: var(--fw-semibold) var(--fs-n2) var(--font); + letter-spacing: 0.02em; + color: var(--code-muted); +} + +.code-title.fname { + font: var(--fw-semibold) var(--fs-n2) var(--mono); + letter-spacing: 0; + color: var(--code-ink); +} + +.code-title + pre { + padding-block: var(--sp-4); + padding-right: var(--sp-5); +} + +/* Copy, centred in the 40px head. */ +.code-title ~ .copy { + top: 6px; + right: 6px; +} + +/* A citation is one long line to read, not a file to keep the shape of. */ +.prose .code.bare[data-lang="apa"] pre { + white-space: pre-wrap; + overflow-wrap: anywhere; +} + +/* A tab-separated file says what Copy keeps. */ +.code.tsv .copy > .say-1::after { + content: " with tabs"; +} + +/* What a command prints: the front page's dashed frame, and nothing to copy. */ +.code.out { + --cover: var(--surface-1); + margin-top: 0; + padding: 0; + background: var(--surface-1); + border: 1px dashed var(--line-2); + color: var(--ink-2); + white-space: normal; +} + +.code.out pre { + color: var(--ink-soft); +} + +.code.out .code-title { + border-bottom: 1px dashed var(--line-2); + color: var(--muted); +} + +.code.out .copy { + display: none; +} + +/* A command and what it prints read as one: 12px apart, not a block's 20. */ +.prose .code.cmd + .code.out { + margin-top: calc(var(--sp-3) - var(--sp-5)); +} + +/* On a phone the docs' blocks sit a little further out to their edges (the + strip above a block with no head is set with the front page's, above). */ +@media (max-width: 600px) { + .prose .code pre:not([data-command], [data-check]) { + padding-right: 16px; + padding-left: 14px; + } + + .code-title { + padding-left: 14px; + } +} + +kbd { + display: inline-block; + min-width: 1.6em; + padding: 0.05em 0.45em; + border: 1px solid var(--line-2); + border-bottom-width: 2px; + border-radius: var(--r-1); + background: var(--surface-3); + font: var(--fw-semibold) 0.8em var(--mono); + text-align: center; + color: var(--ink); +} + +.out { + margin-top: var(--sp-3); + background: var(--surface-1); + border: 1px dashed var(--line-2); + border-radius: var(--r-4); + padding: var(--sp-4) var(--sp-5); + font-family: var(--mono); + font-size: var(--fs-n2); + line-height: var(--lh-read); + color: var(--ink-soft); + white-space: pre; + overflow-x: auto; +} + +.out .ok { + color: var(--ok); + font-weight: var(--fw-bold); +} + +.out .cm { + color: var(--muted); +} + +/* โ”€โ”€ Tabs โ”€โ”€ */ + +.tabs { + display: flex; + gap: var(--sp-1); + margin-bottom: var(--sp-3); + flex-wrap: wrap; +} + +.tab { + border: 1px solid var(--line-2); + background: var(--control); + color: var(--ink-soft); + font: var(--fw-semibold) var(--fs-n2) var(--font); + min-height: 32px; + padding: 0 var(--sp-3); + border-radius: var(--r-3); + cursor: pointer; +} + +@media (hover: hover) { + .tab:hover { + color: var(--ink); + border-color: var(--line-3); + background: color-mix(in srgb, var(--ink-1) 7%, var(--control)); + } +} + +/* Chosen: the one solid accent of a set of choices, as the segments and the + stories are. */ +.tab[aria-selected="true"] { + background: var(--accent); + border-color: var(--accent); + color: var(--on-accent); +} + +.tabpanel[hidden] { + display: none; +} + +/* โ”€โ”€ Command builder โ”€โ”€ */ + +.builder { + display: grid; + grid-template-columns: minmax(0, 0.85fr) minmax(0, 1.15fr); + gap: var(--sp-6); + align-items: start; + margin-top: var(--sp-6); +} + +@media (max-width: 900px) { + .builder { + grid-template-columns: minmax(0, 1fr); + } +} + +/* Without JavaScript the choices do nothing, so they are not shown: the + commands for Docker, written into the page, are. */ +html:not(.js) .builder-qs { + display: none; +} + +/* The same for every control that needs the script, and every drawing, + which the script makes: in its place, a line that says so, and none of + the room kept for it. */ +html:not(.js) :is([data-hero-replay], .hero-segs, [data-fig-switch], .metro-opts, .metro-hint, .metro-info, .how-pick, .how-steps, button.copy, [data-theme-toggle], .docs-search) { + display: none; +} + +html:not(.js) :is([data-hero-stage], [data-fig-stage], [data-metro-stage], [data-stage])::before { + content: "This drawing of real genes is made in the browser: turn JavaScript on to see it."; + display: block; + padding: var(--sp-6); + color: var(--ink-3); + font-size: var(--fs-n2); + line-height: var(--lh-ui); +} + +html:not(.js) .oc-fig::before { + padding: var(--sp-2) 0; +} + +html:not(.js) :is(.case-fig, .oc-fig, .metro-scroll, .how-stage, .dup-fig > .case-fig, .dup-side > .case-fig) { + height: auto; + min-height: 0; + aspect-ratio: auto; +} + +html:not(.js) .builder { + grid-template-columns: minmax(0, 1fr); +} + +.q { + margin-bottom: var(--sp-5); +} + +.q-title { + font-weight: var(--fw-semibold); + margin-bottom: var(--sp-2); + font-size: var(--fs-n1); + color: var(--ink-1); +} + +.seg { + display: flex; + flex-wrap: wrap; + gap: var(--sp-2); +} + +.seg button { + min-height: 36px; + border: 1px solid var(--line-2); + background: var(--control); + color: var(--ink-1); + font: var(--fw-semibold) var(--fs-n2) var(--font); + padding: 0 var(--sp-4); + border-radius: var(--r-3); + cursor: pointer; +} + +/* A one-of-many choice (role=radio) or a switch between drawings: chosen, + it takes the accent, solid, so what is chosen shows at a glance. Where + the reader is (the current page, the open question) takes the wash. */ +.seg button:is([aria-pressed="true"], [aria-checked="true"]) { + background: var(--accent); + border-color: var(--accent); + color: var(--on-accent); +} + +@media (hover: hover) { + .seg button:not([aria-pressed="true"], [aria-checked="true"]):hover { + border-color: var(--line-3); + background: color-mix(in srgb, var(--ink-1) 7%, var(--control)); + } +} + +.opt { + display: grid; + grid-template-columns: auto 1fr; + gap: 2px var(--sp-3); + align-items: start; + padding: var(--sp-3) var(--sp-4); + border: 1px solid var(--line-2); + border-radius: var(--r-3); + background: var(--control); + margin-bottom: var(--sp-2); + cursor: pointer; +} + +@media (hover: hover) { + .opt:hover { + border-color: var(--line-3); + } +} + +.opt:has(input:checked) { + border-color: var(--accent-line); + background: var(--accent-wash); +} + +.opt input[type="checkbox"] { + width: 18px; + height: 18px; + margin: 2px 0 0; + accent-color: var(--accent); + cursor: inherit; +} + +.opt b { + font-weight: var(--fw-semibold); + font-size: var(--fs-n1); +} + +.opt small { + grid-column: 2; + color: var(--ink-3); + font-size: var(--fs-n2); + line-height: var(--lh-ui); +} + +/* An option that does not apply to the choices above, such as a laptop's + ceilings on a cluster. */ +.opt:has(input:disabled) { + cursor: not-allowed; + border-style: dashed; + border-color: var(--line-2); + background: var(--control); +} + +.opt:has(input:disabled) input { + opacity: 0.6; +} + +.opt:has(input:disabled) b { + color: var(--muted); +} + +/* The two commands: the check, then the run. */ +.cmd-head { + margin: 0 0 var(--sp-2); + font-size: var(--fs-n1); + font-weight: var(--fw-semibold); +} + +.cmd-head ~ .cmd-head { + margin-top: var(--sp-6); +} + +.cmd-head code { + white-space: nowrap; +} + +.builder-out > .note { + margin-top: 10px; +} + +.builder-out > .out { + margin-top: 8px; +} + +.builder-notes p + p { + margin-top: 6px; +} + +/* What each line does. Every flag shares one column, as wide as the + longest one needs. */ +.explain { + margin: 12px 0 0; + padding: 0; + list-style: none; + display: grid; + grid-template-columns: fit-content(15em) minmax(0, 1fr); + gap: var(--sp-2) var(--sp-4); + font-size: var(--fs-n2); +} + +.explain li { + grid-column: 1 / -1; + display: grid; + grid-template-columns: subgrid; + align-items: baseline; + color: var(--ink-soft); +} + +@supports not (grid-template-columns: subgrid) { + .explain li { + grid-template-columns: minmax(150px, max-content) minmax(0, 1fr); + column-gap: 12px; + } +} + +.explain .flags { + display: flex; + flex-wrap: wrap; + gap: 4px; + align-self: start; +} + +.explain code { + white-space: nowrap; +} + +@media (max-width: 560px) { + .q { + margin-bottom: var(--sp-4); + } + + .opt { + padding: var(--sp-2) var(--sp-3); + margin-bottom: 6px; + } + + .opt small { + font-size: var(--fs-n2); + } + + .explain { + grid-template-columns: minmax(0, 1fr); + } + + .explain li { + row-gap: 2px; + } +} + +/* โ”€โ”€ What AMAP decides: the five outcomes, one figure โ”€โ”€ */ + +/* Figure 2: one plate, five panels lettered a to e, hairlines between them + rather than five boxes. The conflicts take a row of their own. The docs + draw the same windows one by one, each a plate (.fig.case). */ +.cases { + display: grid; + grid-template-columns: repeat(2, minmax(0, 1fr)); + background: var(--surface-2); + border: 1px solid var(--line-2); + border-radius: var(--r-5); + box-shadow: var(--shadow-2); + counter-reset: panel; +} + +.case { + background: var(--surface-2); + border: 1px solid var(--line-2); + border-radius: var(--r-5); + box-shadow: var(--shadow-2); + padding: var(--sp-6) var(--sp-6) var(--sp-5); + display: flex; + flex-direction: column; + gap: var(--sp-3); + min-width: 0; +} + +.cases > .case { + background: none; + border: 0; + border-radius: 0; + box-shadow: none; + padding: var(--sp-6) var(--plate-pad); +} + +.cases > .case:nth-child(n + 3) { + border-top: 1px solid var(--line-1); +} + +.cases > .case:nth-child(2), +.cases > .case:nth-child(5) { + border-left: 1px solid var(--line-1); +} + +.case-wide { + grid-column: 1 / -1; +} + +.case-head { + display: flex; + align-items: center; + gap: var(--sp-3); + flex-wrap: wrap; +} + +/* The panel's letter, drawn (not read). */ +.cases .case-head::before { + counter-increment: panel; + content: counter(panel, lower-alpha); + content: counter(panel, lower-alpha) / ""; + font-size: var(--fs-1); + font-weight: var(--fw-bold); + line-height: 1; + color: var(--ink-1); +} + +.case-tag { + display: inline-flex; + align-items: center; + gap: 6px; + min-height: 22px; + font: var(--fw-bold) var(--fs-n3)/1 var(--font); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + /* The tracking after the last letter counts as room on its right. */ + padding: 0 calc(var(--sp-2) - var(--tr-caps)) 0 var(--sp-2); + border-radius: var(--r-pill); + border: 1px solid; + background: color-mix(in srgb, currentColor 9%, transparent); +} + +.case-tag.c1 { + color: var(--ink-3); + border-color: var(--line-3); +} + +.case-tag.c2, +.case-tag.c3 { + color: var(--ok); + border-color: color-mix(in srgb, var(--ok) 55%, transparent); +} + +.case-tag.c4 { + color: var(--warn); + border-color: color-mix(in srgb, var(--warn) 55%, transparent); +} + +.case-tag.cm { + color: var(--ref); + border-color: color-mix(in srgb, var(--ref) 55%, transparent); +} + +.case-head h3 { + font-size: var(--fs-1); +} + +.case > p, +.case-caps > p { + color: var(--ink-2); + font-size: var(--fs-n1); + line-height: var(--lh-read); +} + +/* The captions a switcher chooses between share one cell, so the tallest + sets the card's height and a switch does not move the page. */ +.case-caps { + display: grid; +} + +.case-caps > * { + grid-area: 1 / 1; +} + +.case-caps > [hidden] { + display: block; + visibility: hidden; +} + +.case .keeps { + display: flex; + gap: var(--sp-2); + align-items: baseline; + font-size: var(--fs-n1); + color: var(--ink-2); + padding-top: var(--sp-3); + border-top: 1px dashed var(--line-2); + margin-top: auto; +} + +.case .keeps b { + color: var(--amap); + white-space: nowrap; +} + +.case .example { + font-size: var(--fs-n2); + color: var(--muted); +} + +.case .seg button { + min-height: 30px; + font-size: var(--fs-n2); + padding: 0 var(--sp-3); +} + +.case-fig { + position: relative; + min-height: 60px; +} + +/* Panels side by side draw their tracks on the same lines: what room a + shorter text leaves goes above the drawing (and what it keeps, which + follows it to the panel's foot), and a drawing kept as tall as the + tallest a switcher can show sits at the foot of that room. */ +.cases > .case > .case-fig { + margin-top: auto; +} + +.cases > .case > .case-fig + .keeps { + margin-top: 0; +} + +.fig-svg { + display: flex; + flex-direction: column; + justify-content: flex-end; +} + +.cases-hint { + display: none; + margin: calc(-1 * var(--sp-2)) 0 var(--sp-3); + font-size: var(--fs-n2); + color: var(--ink-3); +} + +@media (max-width: 860px) { + .cases { + grid-template-columns: minmax(0, 1fr); + } + + .cases > .case:nth-child(n + 2) { + border-top: 1px solid var(--line-1); + } + + .cases > .case:nth-child(2), + .cases > .case:nth-child(5) { + border-left: 0; + } +} + +/* On a phone the five panels are a strip to swipe, each a plate of its own, + the next one showing at the edge, rather than two thousand pixels to + scroll past. */ +@media (max-width: 600px) { + .cases-hint { + display: block; + } + + .cases { + background: none; + border: 0; + border-radius: 0; + box-shadow: none; + grid-template-columns: none; + grid-auto-flow: column; + grid-auto-columns: 86%; + gap: var(--sp-3); + margin: 0 -16px; + padding: 0 16px var(--sp-3); + overflow-x: auto; + overscroll-behavior-x: contain; + scroll-snap-type: x mandatory; + scroll-padding: 0 16px; + /* Every card as tall as the tallest, what each keeps at its foot. */ + align-items: stretch; + } + + .cases > .case, + .cases > .case:nth-child(n + 2) { + background: var(--surface-2); + border: 1px solid var(--line-2); + border-radius: var(--r-5); + box-shadow: var(--shadow-2); + padding: var(--sp-5) var(--sp-4) var(--sp-4); + scroll-snap-align: start; + } + + /* The strip is no plate; each card is, corners and all. */ + .cases::after { + content: none; + } + + .cases > .case::after { + content: ""; + } + + .case-wide { + grid-column: auto; + } +} + +/* โ”€โ”€ The metro map โ”€โ”€ */ + +.metro-wrap { + background: var(--surface-2); + border: 1px solid var(--line-2); + border-radius: var(--r-5); + box-shadow: var(--shadow-2); + padding: var(--sp-5) var(--plate-pad); +} + +.metro-opts { + display: flex; + flex-wrap: wrap; + gap: var(--sp-2); + align-items: center; + margin-bottom: var(--sp-3); + font-size: var(--fs-n2); + color: var(--ink-3); +} + +.metro-opts label { + display: inline-flex; + align-items: center; + gap: 7px; + min-height: 36px; + padding: 0 var(--sp-3); + border: 1px solid var(--line-2); + border-radius: var(--r-3); + background: var(--control); + cursor: pointer; + color: var(--ink-soft); +} + +@media (hover: hover) { + .metro-opts label:hover { + border-color: var(--line-3); + } +} + +.metro-opts label:has(input:checked) { + border-color: var(--accent-line); + background: var(--accent-wash); + color: var(--ink-1); +} + +.metro-opts input { + accent-color: var(--accent); + margin: 0; + cursor: inherit; +} + +.metro-opts code { + font-size: var(--fs-n2); +} + +/* Under 375px the longest switch, --skip_functional_annotation, keeps to + one line: a step smaller, in a plate and a chip a little less padded. */ +@media (max-width: 374px) { + .metro-wrap { + padding-inline: var(--sp-4); + } + + .metro-opts label { + padding-inline: var(--sp-2); + } + + .metro-opts code { + font-size: var(--fs-n3); + } +} + +/* On a narrow screen the map keeps its size and scrolls sideways, rather + than shrinking its names until they cannot be read. A fade at an edge + says there is more of it that way. */ +.metro-scroll { + overflow-x: auto; + -webkit-overflow-scrolling: touch; +} + +.metro-scroll.more-right { + -webkit-mask-image: linear-gradient(90deg, #000 calc(100% - 48px), transparent); + mask-image: linear-gradient(90deg, #000 calc(100% - 48px), transparent); +} + +.metro-scroll.more-left { + -webkit-mask-image: linear-gradient(90deg, transparent, #000 48px); + mask-image: linear-gradient(90deg, transparent, #000 48px); +} + +.metro-scroll.more-left.more-right { + -webkit-mask-image: linear-gradient(90deg, transparent, #000 48px, #000 calc(100% - 48px), transparent); + mask-image: linear-gradient(90deg, transparent, #000 48px, #000 calc(100% - 48px), transparent); +} + +/* The map's room, kept before the script draws it. */ +.metro-scroll:empty { + aspect-ratio: 1100 / 380; +} + +@media (max-width: 860px) { + .metro-scroll .metro { + min-width: 1000px; + } + + .metro-scroll:empty { + aspect-ratio: auto; + height: 346px; + } +} + +/* In a column of text the map is narrower than its names can bear. */ +.prose .metro-scroll .metro { + min-width: 1000px; +} + +.prose .metro-scroll:empty { + aspect-ratio: auto; + height: 346px; +} + +.metro-hint { + display: none; + font-size: var(--fs-n2); + color: var(--muted); + margin: 0 0 4px; +} + +.prose .metro-hint { + display: block; +} + +/* Where the margin beside the column has room, the plate runs into it and + holds the whole map: nothing to scroll. */ +@media (min-width: 1620px) { + .prose .fig.metro-wrap { + margin-right: -202px; + } + + .prose .metro-hint { + display: none; + } +} + +@media (max-width: 860px) { + .metro-hint { + display: block; + } +} + +.metro { + display: block; + width: 100%; + height: auto; + font-family: var(--font); +} + +.metro .ml { + fill: none; + stroke-width: 7; + stroke-linecap: round; + stroke-linejoin: round; + transition: opacity var(--dur-emph) var(--ease), stroke var(--dur-emph) var(--ease); +} + +.metro .ml-main, +.metro .ml-fn { + stroke: var(--m-main); +} + +.metro .ml-ref { + stroke: var(--m-ref); +} + +.metro .ml-db { + stroke: var(--m-db); +} + +.metro .ml-dup { + stroke: var(--m-dup); +} + +.metro .ml-pan { + stroke: var(--m-pan); +} + +.metro .ml-rep, +.metro .ml-rep-feed { + stroke: var(--m-main); +} + +.metro .stn { + cursor: pointer; + transition: opacity var(--dur-emph) var(--ease); +} + +.metro .stn .dot { + fill: var(--halo); + stroke: var(--ink); + stroke-width: 3; +} + +.metro .stn .hub { + fill: var(--halo); + stroke: var(--ink); + stroke-width: 3; +} + +.metro .stn .end { + fill: var(--ink); +} + +/* A halo of the card's colour, so a line that crosses a name never cuts it. */ +.metro .stn text { + paint-order: stroke; + stroke: var(--halo); + stroke-width: 4px; + stroke-linejoin: round; +} + +.metro .stn text.name { + fill: var(--ink); + font-size: 13px; + font-weight: var(--fw-bold); +} + +.metro .stn text.proc { + fill: var(--muted); + font-size: 11px; + font-family: var(--mono); +} + +/* Picked: filled with the accent. Focused: a heavy ring of ink, so the two + never look alike. */ +.metro .stn.on .dot, +.metro .stn.on .hub { + fill: var(--accent); + stroke: var(--accent); +} + +@media (hover: hover) { + .metro .stn:hover .dot, + .metro .stn:hover .hub { + fill: var(--accent); + stroke: var(--accent); + } +} + +.metro .stn:focus { + outline: none; +} + +.metro .stn:focus-visible .dot, +.metro .stn:focus-visible .hub, +.metro .stn:focus-visible .end { + stroke: var(--ink); + stroke-width: 4; +} + +.metro .stn:focus-visible text.name { + text-decoration: underline; + text-decoration-thickness: 2px; + text-underline-offset: 3px; +} + +.metro .stn.on text.name { + fill: var(--accent); +} + +.metro .exit { + font-size: 10.5px; + font-family: var(--mono); + fill: var(--muted); + transition: opacity var(--dur-emph) var(--ease); +} + +.metro .exit-tick { + stroke: var(--line-strong); + stroke-width: 1.5; + stroke-dasharray: 2 2; +} + +.metro .off { + opacity: 0.16; +} + +.metro .stn.off { + opacity: 0.3; +} + +.metro .skipped text.name { + text-decoration: line-through; +} + +.metro-info { + display: grid; + grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); + gap: var(--sp-3) var(--sp-6); + margin-top: var(--sp-3); + padding: var(--sp-4) var(--sp-5); + border-radius: var(--r-4); + background: var(--surface-0); + border: 1px solid var(--line-1); + min-height: 112px; +} + +@media (max-width: 760px) { + .metro-info { + grid-template-columns: minmax(0, 1fr); + } +} + +.metro-info h3 { + font-size: var(--fs-0); + margin-bottom: 4px; +} + +.metro-info h3 code { + font-size: var(--fs-n2); + margin-left: 6px; + color: var(--muted); +} + +.metro-info p { + font-size: var(--fs-n1); + color: var(--ink-soft); +} + +/* Kept tall for the longest station, the panel reads from its top. */ +.metro-info { + align-items: start; +} + +.metro-info dl { + margin: 0; + align-content: start; + display: grid; + grid-template-columns: max-content minmax(0, 1fr); + gap: 4px 12px; + font-size: var(--fs-n2); +} + +.metro-info dt { + color: var(--muted); + font-weight: var(--fw-semibold); +} + +.metro-info dd { + margin: 0; + color: var(--ink-soft); +} + +.metro-info .hint { + color: var(--muted); + font-size: var(--fs-n1); + align-self: center; +} + +.metro-writes { + margin-top: var(--sp-3); + font-size: var(--fs-n2); + color: var(--ink-3); +} + +.metro-writes code { + font-size: var(--fs-n2); +} + +.metro-lines { + display: flex; + flex-wrap: wrap; + gap: var(--sp-2) var(--sp-5); + margin: var(--sp-2) 0 var(--sp-1); + font-size: var(--fs-n2); + color: var(--ink-3); +} + +.metro-lines span { + display: inline-flex; + align-items: center; + gap: 7px; + transition: opacity var(--dur-emph) var(--ease); +} + +/* A line the switches leave out, faded in the key as on the map. */ +.metro-lines .off { + opacity: 0.35; +} + +.metro-lines i { + width: 20px; + height: 5px; + border-radius: 3px; + display: inline-block; +} + +/* โ”€โ”€ Output cards โ”€โ”€ */ + +/* Six files, as open columns rather than boxes: each under a rule in the + colour of the metro line of the step that writes it, the sample's alone + for what every run writes, joined by the eggNOG database's for what only + a full run writes. */ +.files { + display: grid; + grid-template-columns: repeat(3, minmax(0, 1fr)); + gap: var(--sp-8) var(--sp-8); +} + +@media (max-width: 980px) { + .files { + grid-template-columns: repeat(2, minmax(0, 1fr)); + } +} + +@media (max-width: 520px) { + .files { + grid-template-columns: minmax(0, 1fr); + gap: var(--sp-5); + } +} + +.file { + position: relative; + padding-top: var(--sp-5); + display: flex; + flex-direction: column; + gap: var(--sp-2); +} + +.file .lane { + position: absolute; + top: 0; + left: 0; + right: 0; + display: block; + height: 3px; + border-radius: 2px; + background: var(--m-main); +} + +.file.full-run .lane { + background: linear-gradient(90deg, var(--m-main) 0 50%, var(--m-db) 50% 100%); +} + +/* A path breaks where it has a , after its folder, before anywhere else. */ +.file code { + font-size: var(--fs-n2); + overflow-wrap: anywhere; + background: none; + border: none; + padding: 0; + color: var(--ink-1); + font-weight: var(--fw-semibold); +} + +.file code span { + white-space: nowrap; +} + +.file p { + font-size: var(--fs-n1); + color: var(--ink-2); + line-height: var(--lh-read); +} + +.file .when { + margin-top: auto; + padding-top: var(--sp-1); + font-size: var(--fs-n2); + color: var(--ink-3); +} + +/* On a phone a file is a row: the line down its side, then the file, what + it is and when it is written. */ +@media (max-width: 520px) { + .file { + padding: 0 0 0 var(--sp-4); + } + + .file .lane { + top: 2px; + bottom: 2px; + right: auto; + width: 3px; + height: auto; + } + + .file.full-run .lane { + background: linear-gradient(180deg, var(--m-main) 0 50%, var(--m-db) 50% 100%); + } + + .file p { + font-size: var(--fs-n2); + line-height: 1.45; + } + + /* Each file says when it is written; the colour key can go. */ + .files-key { + display: none; + } +} + +/* Real lines of a real run, with the marks AMAP writes picked out. */ +.excerpt { + margin-top: var(--sp-6); + background: var(--surface-2); + border: 1px solid var(--line-2); + border-radius: var(--r-5); + box-shadow: var(--shadow-2); + padding: var(--sp-5) var(--plate-pad); +} + +.excerpt-head { + display: flex; + flex-wrap: wrap; + justify-content: space-between; + gap: var(--sp-2) var(--sp-4); + align-items: center; + margin-bottom: var(--sp-3); +} + +.excerpt-head p { + font-size: var(--fs-n2); + color: var(--muted); +} + +.excerpt .tabs { + margin-bottom: 0; +} + +.excerpt pre { + white-space: pre-wrap; + overflow-wrap: anywhere; + margin: 0; + padding: var(--sp-4) var(--sp-5); + border-radius: var(--r-4); + background: var(--code-bg); + border: 1px solid var(--code-border); + color: var(--code-ink); + font: var(--fs-n2)/1.6 var(--mono); + overflow-x: auto; +} + +/* On a phone a GFF3 or GenBank line wrapped into ten is no longer a line: + they keep their shape and scroll. */ +@media (max-width: 600px) { + .excerpt { + padding: var(--sp-3); + } + + .excerpt pre { + white-space: pre; + overflow-wrap: normal; + /* The code blocks' scroll shadows: a shadow shows on a side only where + there is more to see. */ + background: + linear-gradient(to right, var(--code-bg) 40%, transparent) left / 28px 100% no-repeat local, + linear-gradient(to left, var(--code-bg) 40%, transparent) right / 28px 100% no-repeat local, + radial-gradient(farthest-side at 0 50%, var(--excerpt-shade), transparent) left / 12px 100% no-repeat scroll, + radial-gradient(farthest-side at 100% 50%, var(--excerpt-shade), transparent) right / 12px 100% no-repeat scroll, + var(--code-bg); + } +} + +.excerpt { + --excerpt-shade: rgba(255, 255, 255, 0.18); +} + +:root[data-theme="light"] .excerpt { + --excerpt-shade: rgba(16, 27, 24, 0.2); +} + +.excerpt mark { + background: color-mix(in srgb, var(--pseudo) 22%, transparent); + color: inherit; + border-radius: 3px; + padding: 0 2px; +} + +.excerpt mark.part { + background: color-mix(in srgb, var(--info) 24%, transparent); +} + +.excerpt mark.name { + background: color-mix(in srgb, var(--amap) 22%, transparent); +} + +.excerpt .key { + display: flex; + flex-wrap: wrap; + gap: var(--sp-2) var(--sp-5); + margin-top: var(--sp-3); + font-size: var(--fs-n2); + color: var(--ink-3); +} + +.excerpt .key mark { + font-family: var(--mono); + font-size: var(--fs-n2); + color: var(--ink); +} + +/* The case counts: the terminal prints what the command prints, and what + each row means sits outside it, in a column of its own. */ +.counts { + display: grid; + gap: 12px; +} + +.excerpt .counts-term { + white-space: pre-wrap; + overflow-wrap: normal; +} + +.counts-term .ln { + display: block; + padding-left: 4ch; + text-indent: -4ch; +} + +.counts-term :is(.t, .f, .v) { + white-space: nowrap; +} + +/* A path longer than a phone breaks after one of its folders. */ +.counts-term .path { + white-space: normal; +} + +.counts-term .p { + color: var(--code-muted); + user-select: none; +} + +.counts-term .f { + color: var(--code-flag); +} + +.counts-term .v { + color: var(--code-value); +} + +.gloss { + font-size: var(--fs-n2); + color: var(--muted); +} + +.gloss-head { + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + margin-bottom: 4px; +} + +.gloss dl { + display: grid; + gap: 2px; + margin: 0; +} + +.gloss dl div { + display: grid; + grid-template-columns: 9.5em minmax(0, 1fr); + gap: 10px; +} + +.gloss dt { + font-family: var(--mono); + color: var(--ink-soft); +} + +.gloss dd { + margin: 0; +} + +/* Wide enough for both: each meaning on the line of the row it explains. The + gloss starts a line down, past the command, and keeps the terminal's + line height; the row names are already in the terminal. */ +@media (min-width: 1060px) { + .counts { + grid-template-columns: max-content minmax(0, 1fr); + gap: 0 20px; + align-items: start; + } + + .gloss { + --line-h: calc(0.8rem * 1.6); + padding-top: calc(15px + var(--line-h)); + line-height: var(--line-h); + } + + .gloss-head { + margin: 0; + line-height: var(--line-h); + } + + .gloss dl { + gap: 0; + } + + .gloss dl div { + display: block; + white-space: nowrap; + } + + .gloss dt { + position: absolute; + width: 1px; + height: 1px; + overflow: hidden; + clip-path: inset(50%); + } +} + +/* โ”€โ”€ Report preview โ”€โ”€ */ + +.report-frame { + border: 1px solid var(--line-2); + border-radius: var(--r-5); + box-shadow: var(--shadow); + background: var(--surface-2); +} + +.report-bar { + position: relative; + display: flex; + align-items: center; + gap: var(--sp-2); + min-height: 44px; + padding: 0 var(--plate-pad); + border-bottom: 1px solid var(--line-1); + font-size: var(--fs-n2); + color: var(--ink-3); +} + +/* On the narrowest phones the window's dots go and the link takes a line of + its own when the name and it do not fit side by side. */ +@media (max-width: 420px) { + .report-bar { + flex-wrap: wrap; + row-gap: 2px; + padding: var(--sp-2) var(--sp-3); + } + + .report-bar i { + display: none; + } + + .report-bar > span { + margin-left: 0; + } +} + +.report-bar i { + width: 10px; + height: 10px; + border-radius: 50%; + background: var(--line-strong); + display: inline-block; +} + +.report-bar > span { + margin-left: 6px; + font-family: var(--mono); +} + +.report-bar a { + margin-left: auto; + font-weight: var(--fw-semibold); + font-size: var(--fs-n2); + white-space: nowrap; +} + +/* Says that what it sits next to was made from synthetic data. */ +.tag-synthetic { display: inline-flex; align-items: center; - gap: 6px; - font: 750 0.74rem var(--font); - letter-spacing: 0.05em; - text-transform: uppercase; - padding: 3px 9px; - border-radius: 999px; - border: 1px solid; + padding: 2px 8px; + border: 1px solid color-mix(in srgb, var(--note) 60%, transparent); + border-radius: var(--r-pill); + background: color-mix(in srgb, var(--note) 14%, transparent); + color: var(--ink-soft); + font: var(--fw-semibold) var(--fs-n3)/1.35 var(--font); + font-style: normal; + letter-spacing: 0.02em; + white-space: nowrap; } -.case-tag.c1 { - color: var(--muted); - border-color: var(--line-strong); +/* Until the reader asks, the stage is a picture of the report's first + screen (tools/pictures.mjs takes it): the report itself is 1.9 MB of + scripts and plots. Once asked, it loads over the picture and fades in. */ +/* The frame's last row: its corners are the frame's, less its edge. */ +.report-stage { + position: relative; + height: 680px; + background: var(--bg-subtle); + overflow: hidden; + border-radius: 0 0 calc(var(--r-5) - 1px) calc(var(--r-5) - 1px); } -.case-tag.c2, -.case-tag.c3 { - color: var(--ok); - border-color: color-mix(in srgb, var(--ok) 55%, transparent); +.report-poster { + display: block; + width: 100%; + height: 100%; + object-fit: cover; + object-position: left top; } -.case-tag.c4 { - color: var(--warn); - border-color: color-mix(in srgb, var(--warn) 55%, transparent); +.report-frame iframe { + position: absolute; + inset: 0; + display: block; + width: 100%; + height: 100%; + border: 0; + background: #fff; + opacity: 0; + transition: opacity var(--dur-emph) var(--ease); } -.case-tag.cm { - color: var(--ref); - border-color: color-mix(in srgb, var(--ref) 55%, transparent); +.report-stage.loaded iframe { + opacity: 1; } -.case-head h3 { - font-size: 1.12rem; +.report-stage:not(.live) iframe { + pointer-events: none; +} + +/* In the dark theme the white report would be the brightest thing on the + page: dimmed until it is opened. */ +:root:not([data-theme="light"]) .report-stage:not(.live) :is(.report-poster, iframe) { + filter: brightness(0.78) saturate(0.9); +} + +.report-cover { + position: absolute; + inset: 0; + display: grid; + place-items: end center; + padding: 0 16px 28px; + border: 0; + background: none; + color: inherit; + cursor: pointer; + font: inherit; +} + +/* A frosted band behind the button, so it never sits on the report's text. */ +.report-cover::before { + content: ""; + position: absolute; + left: 0; + right: 0; + bottom: 0; + height: 104px; + border-top: 1px solid var(--line-1); + background: color-mix(in srgb, var(--surface-2) 90%, transparent); + -webkit-backdrop-filter: blur(6px); + backdrop-filter: blur(6px); +} + +/* In the light theme the band is a step darker than the white report, so + it shows as a bar there as it does in the dark. */ +:root[data-theme="light"] .report-cover::before { + background: color-mix(in srgb, var(--surface-3) 92%, transparent); + border-top-color: var(--line-2); +} + +.report-cover span { + position: relative; + background: var(--accent); + color: var(--on-accent); + display: inline-flex; + align-items: center; + min-height: 44px; + padding: 0 var(--sp-5); + border-radius: var(--r-3); + font-weight: var(--fw-semibold); + box-shadow: var(--shadow); +} + +@media (hover: hover) { + .report-cover:hover span { + background: var(--accent-hover); + } +} + +/* The ring goes round the button, on the frosted band: on the dimmed + poster no colour would stand out. */ +.report-cover:focus-visible { + outline: none; +} + +.report-cover:focus-visible span { + outline: 3px solid var(--accent); + outline-offset: 3px; +} + +.report-stage.live .report-cover { + display: none; +} + +/* On a touch screen nothing leaves the report the way a mouse does: Done, + in the window's bar beside the link, gives the page its swipes back. It + sits over none of the report. */ +.report-done { + display: none; + margin-left: auto; + padding: var(--sp-2) var(--sp-4); + border: 1px solid var(--line-strong); + border-radius: var(--r-pill); + background: var(--bg-elev); + color: var(--ink); + font: var(--fw-semibold) var(--fs-n2) var(--font); + cursor: pointer; +} + +@media (hover: none) { + .report-frame:has(.report-stage.live) .report-done:not([hidden]) { + display: inline-flex; + } + + .report-frame:has(.report-stage.live) .report-done:not([hidden]) + a { + margin-left: var(--sp-2); + } } -.case > p { +.report-mobile { + display: none; + padding: 32px 20px; + text-align: center; color: var(--ink-soft); - font-size: 0.95rem; } -.case .keeps { +@media (max-width: 700px) { + .report-stage { + display: none; + } + + .report-mobile { + padding: 24px 16px; + } + + .report-bar .tag-synthetic { + display: none; + } + + .report-mobile { + display: grid; + gap: 12px; + justify-items: center; + } +} + +/* โ”€โ”€ Where the project lives โ”€โ”€ */ + +/* The last word before the footer: the accent's wash in a corner, the slate + of the logo in the other. */ +.cta { + background: + radial-gradient(55% 140% at 0% 0%, color-mix(in srgb, var(--brand-teal) 18%, transparent), transparent 70%), + radial-gradient(50% 140% at 100% 100%, color-mix(in srgb, var(--brand-slate) 14%, transparent), transparent 70%); +} + +:root[data-theme="light"] .cta { + background: + radial-gradient(55% 140% at 0% 0%, color-mix(in srgb, var(--brand-aqua) 12%, transparent), transparent 70%), + radial-gradient(50% 140% at 100% 100%, color-mix(in srgb, var(--brand-slate) 12%, transparent), transparent 70%); +} + +.cta-wrap { display: flex; - gap: 8px; - align-items: baseline; - font-size: 0.92rem; - color: var(--ink-soft); - padding-top: 10px; - border-top: 1px solid var(--line); - margin-top: auto; + flex-wrap: wrap; + align-items: center; + justify-content: space-between; + gap: var(--sp-6); +} + +.cta p { + color: var(--ink-2); + font-size: var(--fs-1); + margin-top: var(--sp-3); +} + +.cta .actions { + margin-top: 0; +} + +/* โ”€โ”€ FAQ โ”€โ”€ */ + +/* The questions read in the main column, as one list between a rule over + it and one under it, a hairline between each two. */ +.faq { + display: grid; + max-width: none; + border-top: 1px solid var(--line-2); + border-bottom: 1px solid var(--line-2); } -.case .keeps b { - color: var(--amap); - white-space: nowrap; +.faq details + details { + border-top: 1px solid var(--line-1); } -.case .example { - font-size: 0.8rem; - color: var(--muted); +.faq summary { + cursor: pointer; + padding: var(--sp-5) 0; + font-size: var(--fs-1); + font-weight: var(--fw-semibold); + letter-spacing: var(--tr-heading); + line-height: var(--lh-ui); + color: var(--ink-1); + list-style: none; + display: flex; + justify-content: space-between; + align-items: flex-start; + gap: var(--sp-4); } -.case .seg button { - font-size: 0.82rem; - padding: 5px 10px; +@media (hover: hover) { + .faq summary:hover { + color: var(--accent); + } } -.case-fig { - position: relative; - min-height: 60px; +@media (hover: hover) { + .faq summary:hover::after { + border-color: var(--accent-line); + color: var(--accent); + } } -/* โ”€โ”€ The metro map โ”€โ”€ */ +.faq summary > span { + flex: 1; +} -.metro-wrap { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 16px; - padding: 16px; +.faq summary::-webkit-details-marker { + display: none; } -.metro-opts { - display: flex; - flex-wrap: wrap; - gap: 8px 10px; - align-items: center; - margin-bottom: 8px; - font-size: 0.86rem; - color: var(--muted); +/* Drawn, not read: the empty alternative text keeps "+" out of the question's + name (a browser without it keeps the first declaration). */ +:is(.faq details, .docs-menu, .toc-inline) > summary::after { + content: "+"; + content: "+" / ""; + flex: none; + display: grid; + place-items: center; + width: 24px; + height: 24px; + margin-top: 1px; + border: 1px solid var(--line-2); + border-radius: 50%; + color: var(--ink-3); + font-weight: var(--fw-medium); + font-size: var(--fs-n1); + line-height: 1; } -.metro-opts label { - display: inline-flex; - align-items: center; - gap: 7px; - padding: 6px 10px; - border: 1px solid var(--line); - border-radius: 9px; - background: var(--bg); - cursor: pointer; - color: var(--ink-soft); +:is(.faq details, .docs-menu, .toc-inline)[open] > summary::after { + content: "\2212"; + content: "\2212" / ""; + border-color: var(--accent-line); + background: var(--accent-wash); + color: var(--accent); } -.metro-opts label:hover { - border-color: var(--line-strong); +@media (max-width: 600px) { + .faq summary { + padding: var(--sp-4) 0; + font-size: var(--fs-0); + } } -.metro-opts input { - accent-color: var(--accent); - margin: 0; +.faq .ans { + max-width: var(--measure); + padding: 0 var(--sp-8) var(--sp-6) 0; + color: var(--ink-2); + font-size: var(--fs-n1); + line-height: var(--lh-read); + display: grid; + gap: var(--sp-3); } -.metro-opts code { - font-size: 0.8rem; +@media (max-width: 600px) { + .faq .ans { + padding-right: 0; + } } -/* On a narrow screen the map keeps its size and scrolls sideways, rather - than shrinking its names until they cannot be read. */ -.metro-scroll { - overflow-x: auto; - -webkit-overflow-scrolling: touch; +/* Measured times, each with what was run. */ +.faq .times { + margin: 0; + padding: 0; + list-style: none; } -@media (max-width: 860px) { - .metro-scroll .metro { - min-width: 1000px; - } +.faq .times li { + display: flex; + justify-content: space-between; + align-items: baseline; + gap: var(--sp-4); + padding: var(--sp-2) 0; + border-bottom: 1px dashed var(--line-2); } -/* In a column of text the map is narrower than its names can bear. */ -.prose .metro-scroll .metro { - min-width: 1000px; +.faq .times b { + flex: none; + color: var(--ink-1); + font-variant-numeric: tabular-nums; } -.metro-hint { - display: none; - font-size: 0.8rem; - color: var(--muted); - margin: 0 0 4px; +/* โ”€โ”€ Footer โ”€โ”€ */ + +/* The base of every page: one level under the page, columns of quiet links + under caps labels, and a bottom row under a hairline. */ +.foot { + border-top: 1px solid var(--line-1); + background: var(--surface-sunk); + padding: var(--sp-10) 0 var(--sp-11); + font-size: var(--fs-n1); + line-height: var(--lh-ui); + color: var(--ink-3); } -.prose .metro-hint { - display: block; +/* A footer that ends in the bottom row needs less room under it. */ +.foot:has(> .foot-base) { + padding-bottom: var(--sp-7); } -@media (max-width: 860px) { - .metro-hint { - display: block; - } +.foot .wrap { + display: grid; + grid-template-columns: 1.4fr 1fr 1fr; + gap: var(--sp-7); } -.metro { - display: block; - width: 100%; - height: auto; - font-family: var(--font); +.foot h3 { + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + line-height: var(--lh-label); + text-transform: uppercase; + letter-spacing: var(--tr-caps); + color: var(--ink-3); + margin-bottom: var(--sp-4); } -.metro .ml { - fill: none; - stroke-width: 7; - stroke-linecap: round; - stroke-linejoin: round; - transition: opacity 0.3s ease, stroke 0.3s ease; +.foot ul { + list-style: none; + margin: 0; + padding: 0; + display: grid; + gap: var(--sp-2); } -.metro .ml-main { - stroke: var(--m-main); +/* Quiet links: the footer is a list of ways out, not a paragraph. */ +.foot a { + color: var(--ink-2); + text-decoration: none; } -.metro .ml-ref { - stroke: var(--m-ref); +@media (hover: hover) { + .foot a:hover { + color: var(--accent); + text-decoration: underline; + } } -.metro .ml-db { - stroke: var(--m-db); +.foot blockquote { + margin: 0; + color: var(--ink-2); + max-width: 30em; } -.metro .ml-dup { - stroke: var(--m-dup); +.foot .doi { + margin: var(--sp-3) 0 0; } -.metro .ml-pan { - stroke: var(--m-pan); +/* The bottom row, under a hairline: who makes it, its licence, its version. */ +.foot .wrap.foot-base { + display: flex; + flex-wrap: wrap; + align-items: baseline; + gap: var(--sp-1) var(--sp-5); + margin-top: var(--sp-8); + font-size: var(--fs-n2); } -.metro .ml-rep, -.metro .ml-rep-feed { - stroke: var(--m-main); +.foot .wrap.foot-base::before { + content: ""; + flex: 1 0 100%; + border-top: 1px solid var(--line-1); + margin-bottom: var(--sp-4); } -.metro .stn { +/* The switch for the / key, at the end of the footer's last line. */ +.slash-toggle { + margin-left: auto; + padding: var(--sp-1) 0; + border: 0; + background: none; + font: inherit; + color: inherit; + text-decoration: underline; + text-decoration-color: color-mix(in srgb, currentColor 45%, transparent); + text-underline-offset: 3px; cursor: pointer; - transition: opacity 0.3s ease; } -.metro .stn .dot { - fill: var(--bg-elev); - stroke: var(--ink); - stroke-width: 3; +@media (hover: hover) { + .slash-toggle:hover { + color: var(--accent); + text-decoration-color: currentColor; + } } -.metro .stn .hub { - fill: var(--bg-elev); - stroke: var(--ink); - stroke-width: 3; +/* A touch phone has no / key to switch, until a keyboard is used on it + (site.js marks the page then). */ +@media (hover: none) and (pointer: coarse) { + html:not(.keys) .slash-toggle { + display: none; + } } -.metro .stn .end { - fill: var(--ink); +/* A docs page has its search box in the sidebar, which stays in view, so + the header does not offer a second one where the sidebar shows. */ +@media (min-width: 861px) { + body.docs .top .search-btn { + display: none; + } } -.metro .stn text.name { - fill: var(--ink); - font-size: 13px; - font-weight: 700; +/* The missing page's own search field is the first thing under its title, + so neither does it where the header's button would show the / key too. */ +@media (min-width: 900px) { + body.lost-page .top .search-btn { + display: none; + } } -.metro .stn text.proc { - fill: var(--muted); - font-size: 9.5px; - font-family: var(--mono); +/* With the / key off, nothing advertises it. */ +.no-slash .search-key { + display: none; } -.metro .stn:hover .dot, -.metro .stn:hover .hub, -.metro .stn:focus .dot, -.metro .stn:focus .hub, -.metro .stn.on .dot, -.metro .stn.on .hub { - fill: var(--accent); - stroke: var(--accent); -} +/* On a phone the citation spans the footer, its two lists of links side by + side under it. */ +@media (max-width: 760px) { + .foot .wrap:not(.foot-base) { + grid-template-columns: repeat(2, minmax(0, 1fr)); + } -.metro .stn:focus { - outline: none; + .foot .wrap:not(.foot-base) > :first-child { + grid-column: 1 / -1; + } } -.metro .stn.on text.name { - fill: var(--accent); +/* โ”€โ”€ Pages other than the front page โ”€โ”€ */ + +.ref-hero { + padding: var(--sp-10) 0 var(--sp-2); } -.metro .exit { - font-size: 10.5px; - font-family: var(--mono); - fill: var(--muted); +.ref-hero h1 { + font-size: var(--fs-title); + font-weight: var(--fw-display); + letter-spacing: var(--tr-display); } -.metro .exit-tick { - stroke: var(--line-strong); - stroke-width: 1.5; - stroke-dasharray: 2 2; +.ref-hero p { + color: var(--ink-2); + margin-top: var(--sp-4); + font-size: var(--fs-1); + line-height: var(--lh-read); + max-width: 42rem; } -.metro .off { - opacity: 0.16; +@media (max-width: 600px) { + .ref-hero { + padding-top: var(--sp-8); + } + + .ref-hero p { + font-size: var(--fs-0); + line-height: var(--lh-body); + } } -.metro .stn.off { - opacity: 0.3; +.ref-section { + padding: var(--sp-10) 0 var(--sp-12); } -.metro .skipped text.name { - text-decoration: line-through; +.ref-section .note + .deep { + margin-top: var(--sp-5); } -.metro-info { - display: grid; - grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); - gap: 12px 24px; - margin-top: 10px; - padding: 14px 16px; - border-radius: 12px; - background: var(--bg); - border: 1px solid var(--line); - min-height: 112px; +/* A table set as a paper sets one: a heavy rule over it and under it, a + hairline under its head and between its rows, no box: it sits on the + page's grid as the text round it does. A table wider than its box scrolls, + and shades the edge it continues past (the same covers and shadows as a + block of code); the covers are the colour of what the table sits on, the + page or a figure's card. site.js marks a table while it scrolls. */ +.table-wrap { + overflow-x: auto; + border-top: 2px solid var(--ink-3); + border-bottom: 2px solid var(--ink-3); + --cover: var(--surface-0); + --shade: rgba(255, 255, 255, 0.16); } -@media (max-width: 760px) { - .metro-info { - grid-template-columns: minmax(0, 1fr); - } +.table-wrap[data-scroll-region], +html:not(.js) .table-wrap { + background: + linear-gradient(to right, var(--cover) 40%, transparent) left / 28px 100% no-repeat local, + linear-gradient(to left, var(--cover) 40%, transparent) right / 28px 100% no-repeat local, + radial-gradient(farthest-side at 0 50%, var(--shade), transparent) left / 12px 100% no-repeat scroll, + radial-gradient(farthest-side at 100% 50%, var(--shade), transparent) right / 12px 100% no-repeat scroll, + var(--cover); } -.metro-info h3 { - font-size: 1.05rem; - margin-bottom: 4px; +.fig .table-wrap { + --cover: var(--surface-1); } -.metro-info h3 code { - font-size: 0.78rem; - margin-left: 6px; - color: var(--muted); +:root[data-theme="light"] .table-wrap { + --shade: rgba(16, 27, 24, 0.16); } -.metro-info p { - font-size: 0.92rem; - color: var(--ink-soft); +table { + border-collapse: collapse; + width: 100%; + font-size: var(--fs-n1); + line-height: var(--lh-read); + font-variant-numeric: tabular-nums; } -.metro-info dl { - margin: 0; - display: grid; - grid-template-columns: max-content minmax(0, 1fr); - gap: 4px 12px; - font-size: 0.86rem; +th, +td { + text-align: left; + padding: var(--sp-3) var(--sp-4); + border-bottom: 1px solid var(--line-1); + vertical-align: top; } -.metro-info dt { - color: var(--muted); - font-weight: 650; +/* The first and last columns meet the rules' ends; a row opened from a + link keeps room for its bar. */ +.table-wrap :is(th, td):first-child { + padding-left: 0; } -.metro-info dd { - margin: 0; - color: var(--ink-soft); +.table-wrap :is(th, td):last-child { + padding-right: 0; } -.metro-info .hint { - color: var(--muted); - font-size: 0.92rem; - align-self: center; +.table-wrap tr:target > td:first-child { + padding-left: var(--sp-3); } -.metro-writes { - margin-top: 10px; - font-size: 0.86rem; - color: var(--muted); +td { + padding-block: var(--sp-4); + vertical-align: baseline; } -.metro-writes code { - font-size: 0.8rem; +@media (hover: hover) { + tbody tr:hover > td { + background: color-mix(in srgb, var(--ink-1) 2.5%, transparent); + } } -.metro-lines { - display: flex; - flex-wrap: wrap; - gap: 6px 16px; - margin: 8px 0 2px; - font-size: 0.8rem; - color: var(--muted); +th { + font-size: var(--fs-n3); + line-height: var(--lh-label); + text-transform: uppercase; + letter-spacing: var(--tr-caps); + color: var(--ink-3); + border-bottom-color: var(--line-3); + font-weight: var(--fw-bold); + vertical-align: bottom; } -.metro-lines span { - display: inline-flex; - align-items: center; - gap: 7px; +tr:last-child td { + border-bottom: none; } -.metro-lines i { - width: 20px; - height: 5px; - border-radius: 3px; - display: inline-block; +td.num, +th.num { + text-align: right; + font-variant-numeric: tabular-nums; } -/* โ”€โ”€ Output cards โ”€โ”€ */ +/* A number stays whole; the name over its column may wrap. */ +td.num { + white-space: nowrap; +} -.files { +.deep { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); - gap: 14px; + gap: var(--sp-3); } -@media (max-width: 980px) { - .files { - grid-template-columns: repeat(2, minmax(0, 1fr)); +@media (max-width: 860px) { + .deep { + grid-template-columns: minmax(0, 1fr); } } -@media (max-width: 520px) { - .files { - grid-template-columns: minmax(0, 1fr); +/* A last card alone on its row takes the whole row, rather than sit beside + two empty cells. */ +@media (min-width: 861px) { + .deep > a:last-child:nth-child(3n + 1) { + grid-column: 1 / -1; } } -.file { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: var(--radius); - padding: 18px; - display: flex; - flex-direction: column; - gap: 8px; +.deep a { + display: block; + text-decoration: none; + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); + padding: var(--sp-4) var(--sp-5); + color: var(--ink-1); + transition: border-color var(--dur-state) var(--ease), background-color var(--dur-state) var(--ease); +} + +@media (hover: hover) { + .deep a:hover { + border-color: var(--accent-line); + background: var(--surface-2); + } } -.file .ico { - width: 34px; - height: 34px; - border-radius: 9px; - display: grid; - place-items: center; - background: var(--accent-soft); - color: var(--accent); +.deep a b { + display: block; + margin-bottom: 2px; } -.file .ico svg { - width: 19px; - height: 19px; +.deep a span { + display: block; + font-size: var(--fs-n2); + line-height: var(--lh-ui); + color: var(--ink-3); } -.file code { - font-size: 0.8rem; - word-break: break-all; - background: none; - border: none; - padding: 0; - color: var(--ink); - font-weight: 650; -} +/* โ”€โ”€ Documentation pages โ”€โ”€ */ -.file p { - font-size: 0.9rem; - color: var(--muted); - line-height: 1.45; +/* The docs share the front page's frame: the page list, with the page's own + contents under its entry, then the text column, 848px wide on a large + screen (the sidebar and the gap are four squares of the graticule, so the + column starts on one of its major lines). Header, columns and footer share + its edges. Running text keeps a reading measure inside the column, while + code, tables and drawings use all of it. */ +.docs-layout { + /* The sidebar and its gap make whole squares of the graticule: three up + to 1215px, four from 1216px, so the text column starts on a major line + while the logo, the sidebar and the footer keep theirs. */ + --side: calc(3 * var(--grid-sq) - var(--sp-9)); + display: grid; + grid-template-columns: var(--side) minmax(0, 1fr); + column-gap: var(--sp-9); + padding-top: var(--sp-8); + padding-bottom: var(--sp-12); } -/* Real lines of a real run, with the marks AMAP writes picked out. */ -.excerpt { - margin-top: 22px; - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; - padding: 16px; +.docs-layout > .prose { + max-width: none; } -.excerpt-head { - display: flex; - flex-wrap: wrap; - justify-content: space-between; - gap: 8px 16px; - align-items: center; - margin-bottom: 10px; +@media (min-width: 1216px) { + .docs-layout { + --side: calc(4 * var(--grid-sq) - var(--sp-9)); + } } -.excerpt-head p { - font-size: 0.86rem; - color: var(--muted); +@media (max-width: 860px) { + .wrap.docs-layout, + .wrap.docs-home { + grid-template-columns: minmax(0, 1fr); + gap: var(--sp-5); + padding-top: var(--sp-5); + } } -.excerpt .tabs { - margin-bottom: 0; +.docs-nav { + position: sticky; + top: var(--clear); + align-self: start; + font-size: var(--fs-n1); } -.excerpt pre { - white-space: pre-wrap; - overflow-wrap: anywhere; - margin: 0; - padding: 14px 16px; - border-radius: 10px; - background: var(--code-bg); - border: 1px solid var(--code-border); - color: var(--code-ink); - font: 0.8rem/1.6 var(--mono); - overflow-x: auto; +/* A sticky box stacks on its own, so the sidebar sits above the text it + spreads its search results over. */ +.docs-nav { + z-index: 10; } -.excerpt mark { - background: color-mix(in srgb, var(--pseudo) 22%, transparent); - color: inherit; - border-radius: 3px; - padding: 0 2px; +/* The page list, with the contents in it, scrolls on its own; the search + box does not, so its results can spread over the page instead of being cut + off. site.js keeps the current page and the section being read in view. */ +.docs-menu { + max-height: calc(100vh - 150px); + overflow: auto; + scrollbar-width: thin; } -.excerpt mark.part { - background: color-mix(in srgb, var(--info) 24%, transparent); -} +@media (max-width: 860px) { + .docs-nav { + position: static; + } -.excerpt mark.name { - background: color-mix(in srgb, var(--amap) 22%, transparent); + .docs-menu { + max-height: none; + } } -.excerpt .key { - display: flex; - flex-wrap: wrap; - gap: 6px 18px; - margin-top: 10px; - font-size: 0.82rem; - color: var(--muted); +.docs-search { + position: relative; + margin-bottom: 16px; } -.excerpt .key mark { - font-family: var(--mono); - font-size: 0.78rem; - color: var(--ink); +.docs-search input { + width: 100%; + font: var(--fs-n1) var(--font); + height: 40px; + padding: 0 34px 0 var(--sp-3); + border: 1px solid var(--line-3); + border-radius: var(--r-3); + background: var(--control); + color: var(--ink-1); } -/* โ”€โ”€ Report preview โ”€โ”€ */ - -.report-frame { - border: 1px solid var(--line); - border-radius: 14px; - overflow: hidden; - box-shadow: var(--shadow); - background: var(--bg-elev); +.docs-search input::placeholder { + color: var(--ink-3); + opacity: 1; } -.report-bar { - display: flex; - align-items: center; - gap: 8px; - padding: 10px 14px; - border-bottom: 1px solid var(--line); - font-size: 0.82rem; - color: var(--muted); +/* The field's clear button, drawn in the palette rather than the browser's. */ +.docs-search input::-webkit-search-cancel-button { + -webkit-appearance: none; + appearance: none; + width: 16px; + height: 16px; + margin-left: var(--sp-2); + background-color: var(--ink-3); + -webkit-mask: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M6 6l12 12M18 6 6 18' stroke='%23000' stroke-width='2.4' stroke-linecap='round'/%3E%3C/svg%3E") center / 12px no-repeat; + mask: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M6 6l12 12M18 6 6 18' stroke='%23000' stroke-width='2.4' stroke-linecap='round'/%3E%3C/svg%3E") center / 12px no-repeat; + cursor: pointer; } -.report-bar i { - width: 10px; - height: 10px; - border-radius: 50%; - background: var(--line-strong); - display: inline-block; +@media (hover: hover) { + .docs-search input::-webkit-search-cancel-button:hover { + background-color: var(--ink-1); + } } -.report-bar span { - margin-left: 6px; - font-family: var(--mono); +/* Phones zoom into a field typed in at under 16px; this one stays put. */ +@media (max-width: 860px) { + .docs-search input { + font-size: 16px; + } } -.report-bar a { - margin-left: auto; - font-weight: 600; - font-size: 0.85rem; - white-space: nowrap; +/* The "/" that opens the search, shown in the box until it is used. */ +.search-key { + display: inline-grid; + place-items: center; + min-width: 20px; + height: 20px; + padding: 0 5px; + border: 1px solid var(--line-2); + border-bottom-width: 2px; + border-radius: var(--r-1); + background: var(--surface-3); + color: var(--ink-3); + font: var(--fw-semibold) var(--fs-n3)/1 var(--mono); } -/* Says that what it sits next to was made from synthetic data. */ -.tag-synthetic { - display: inline-flex; - align-items: center; - padding: 2px 8px; - border: 1px solid color-mix(in srgb, var(--note) 60%, transparent); - border-radius: 999px; - background: color-mix(in srgb, var(--note) 14%, transparent); - color: var(--ink-soft); - font: 650 0.72rem/1.35 var(--font); - font-style: normal; - letter-spacing: 0.02em; - white-space: nowrap; +.docs-search > .search-key { + position: absolute; + right: 9px; + top: 50%; + transform: translateY(-50%); + pointer-events: none; } -.report-stage { - position: relative; +.docs-search:focus-within > .search-key, +.docs-search input:not(:placeholder-shown) ~ .search-key { + display: none; } -.report-stage:not(.live) iframe { - pointer-events: none; +@media (hover: none) { + .search-key { + display: none !important; + } } -.report-cover { +.docs-results { position: absolute; - inset: 0; - display: grid; - place-items: end center; - padding-bottom: 28px; - border: 0; - background: linear-gradient(to bottom, transparent 60%, color-mix(in srgb, var(--bg) 65%, transparent)); - cursor: pointer; - font: inherit; -} - -.report-cover span { - background: var(--accent); - color: var(--on-accent); - padding: 11px 18px; - border-radius: 10px; - font-weight: 650; - box-shadow: var(--shadow); + z-index: 20; + left: 0; + right: -160px; + top: calc(100% + 6px); + background: var(--surface-2); + border: 1px solid var(--line-2); + border-radius: var(--r-4); + box-shadow: var(--shadow-3); + padding: 0 6px; + max-height: min(70vh, 600px); + overflow: auto; + overscroll-behavior: contain; } -.report-stage.live .report-cover { - display: none; +@media (max-width: 860px) { + .docs-results { + right: 0; + } } -.report-mobile { - display: none; - padding: 32px 20px; - text-align: center; - color: var(--ink-soft); +/* "15 results in 6 pages", held at the top while the list scrolls. */ +.docs-results .r-count { + position: sticky; + top: 0; + z-index: 1; + margin: 0 -6px 2px; + padding: var(--sp-2) var(--sp-4); + border-bottom: 1px solid var(--line-1); + background: var(--surface-2); + color: var(--ink-3); + font-size: var(--fs-n2); } -.report-frame iframe { +.docs-results a { display: block; - width: 100%; - height: 680px; - border: 0; - background: #fff; + padding: var(--sp-2) var(--sp-3); + border-radius: var(--r-2); + text-decoration: none; + color: var(--ink); } -@media (max-width: 700px) { - .report-stage { - display: none; - } - - .report-bar .tag-synthetic { - display: none; +@media (hover: hover) { + .docs-results a:hover { + background: var(--bg-subtle); } +} - .report-mobile { - display: grid; - gap: 14px; - justify-items: center; - } +/* The result Enter opens: an accent bar, 3:1 and more on the panel. */ +.docs-results a.active, +.docs-results a:focus-visible { + background: var(--accent-soft); + box-shadow: inset 3px 0 0 var(--accent); + outline: none; } -/* โ”€โ”€ Where the project lives โ”€โ”€ */ +/* A page, then up to two of its sections, hung under it. */ +.docs-results .r-page { + margin-top: 4px; +} -.cta { - background: - radial-gradient(60% 120% at 0% 0%, color-mix(in srgb, var(--brand-teal) 20%, transparent), transparent 70%), - radial-gradient(60% 120% at 100% 100%, color-mix(in srgb, var(--brand-slate) 18%, transparent), transparent 70%); +.docs-results .r-title { + font-size: var(--fs-n1); + font-weight: var(--fw-semibold); } -.cta-wrap { - display: flex; - flex-wrap: wrap; - align-items: center; - justify-content: space-between; - gap: 24px; +.docs-results .r-page .r-title { + font-weight: var(--fw-bold); } -.cta p { - color: var(--ink-soft); - margin-top: 8px; +.docs-results .r-meta { + float: right; + margin-left: 10px; + color: var(--muted); + font-size: var(--fs-n3); + line-height: 1.6; } -.cta .actions { - margin-top: 0; +.docs-results .r-sec { + position: relative; + margin-left: 14px; } -/* โ”€โ”€ FAQ โ”€โ”€ */ - -.faq { - display: grid; - gap: 10px; - max-width: 860px; +.docs-results .r-sec::before { + content: ""; + position: absolute; + left: -8px; + top: 0; + bottom: 0; + border-left: 1px solid var(--line-strong); } -.faq details { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 10px; - padding: 0 16px; +.docs-results .r-suggest { + margin: 4px 0; + font-size: var(--fs-n1); } -.faq summary { - cursor: pointer; - padding: 14px 0; - font-weight: 650; - list-style: none; - display: flex; - justify-content: space-between; - gap: 14px; +.docs-results .r-suggest b { + color: var(--accent); } -.faq summary > span { - flex: 1; +.docs-results small { + display: block; + color: var(--muted); + font-size: var(--fs-n2); + line-height: 1.4; + margin-top: 2px; } -.faq summary::-webkit-details-marker { - display: none; +.docs-results mark { + background: color-mix(in srgb, var(--accent) 26%, transparent); + color: var(--ink); + border-radius: 3px; } -.faq summary::after { - content: "+"; - color: var(--muted); - font-weight: 500; - font-size: 1.2rem; - line-height: 1; +/* A flag in a result is one word: its dashes stay with its name. */ +.docs-results .nw { + white-space: nowrap; } -.faq details[open] summary::after { - content: "\2212"; +/* The keys, at the foot of the list on screens that have them. */ +.docs-results .r-foot { + position: sticky; + bottom: 0; + margin: 6px -6px 0; + padding: var(--sp-2) var(--sp-4); + border-top: 1px solid var(--line-1); + background: var(--surface-2); + color: var(--ink-3); + font-size: var(--fs-n3); } -.faq .ans { - padding: 0 0 16px; +.docs-results .r-foot kbd { + font-size: var(--fs-n3); color: var(--ink-soft); - font-size: 0.95rem; - display: grid; - gap: 10px; } -/* โ”€โ”€ Footer โ”€โ”€ */ - -.foot { - border-top: 1px solid var(--line); - padding: 40px 0 56px; - font-size: 0.9rem; - color: var(--muted); +@media (hover: none) { + .docs-results .r-foot { + display: none; + } } -.foot .wrap { - display: grid; - grid-template-columns: 1.4fr 1fr 1fr; - gap: 28px; +/* The page list, scrolled to the current page by site.js, ends with room + so its last page is not cut by the edge. While more of it is below the + edge, the edge fades (site.js marks it); the top stays sharp, where the + current page's entry holds. */ +.docs-menu { + padding-bottom: 8px; } -.foot h3 { - font-size: 0.85rem; - text-transform: uppercase; - letter-spacing: 0.06em; - color: var(--ink-soft); - margin-bottom: 10px; +.docs-menu.more-below { + -webkit-mask-image: linear-gradient(to bottom, #000 calc(100% - var(--sp-8)), transparent); + mask-image: linear-gradient(to bottom, #000 calc(100% - var(--sp-8)), transparent); } -.foot ul { - list-style: none; - margin: 0; - padding: 0; - display: grid; - gap: 6px; +.docs-results .none { + padding: 8px 10px; + color: var(--muted); + font-size: var(--fs-n2); } -.foot blockquote { - margin: 0; - color: var(--ink-soft); +.docs-menu summary { + display: none; } -.foot .doi { - margin: 10px 0 0; -} +@media (max-width: 860px) { + .docs-menu { + border: 1px solid var(--line-2); + border-radius: var(--r-4); + background: var(--surface-1); + padding: var(--sp-1) var(--sp-4) var(--sp-2); + } -@media (max-width: 760px) { - .foot .wrap { - grid-template-columns: minmax(0, 1fr); + .docs-menu summary { + display: flex; + cursor: pointer; + font-weight: var(--fw-semibold); + padding: 8px 0; } -} -/* โ”€โ”€ Pages other than the front page โ”€โ”€ */ + /* Folded, it has as much room under its name as over it. */ + .docs-menu:not([open]) { + padding-bottom: var(--sp-1); + } -.ref-hero { - padding: 48px 0 8px; -} + /* Two search boxes at the top of a page is one too many: the header's + opens the same search, with the page's contents under it. */ + .docs-nav .docs-search { + display: none; + } -.ref-hero h1 { - font-size: clamp(2rem, 4.6vw, 3rem); - font-weight: 800; - letter-spacing: -0.03em; + /* The Docs tab lists its pages as cards; it needs no folded list too. */ + .docs-home > .docs-nav { + display: none; + } } -.ref-hero p { - color: var(--muted); - margin-top: 10px; - font-size: 1.06rem; - max-width: 46em; +/* The groups of pages, numbered as the front page numbers its chapters: the + number in the accent (drawn, not read), a short rule, the name in caps. */ +.docs-menu { + counter-reset: docs-group; } -.ref-section { - padding: 48px 0 8px; +.docs-section { + display: flex; + align-items: center; + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + line-height: var(--lh-label); + text-transform: uppercase; + letter-spacing: var(--tr-caps); + color: var(--ink-3); + margin: var(--sp-3) 0 var(--sp-1); + padding-left: var(--sp-3); } -.table-wrap { - overflow-x: auto; - border: 1px solid var(--line); - border-radius: var(--radius); - background: var(--bg-elev); +.docs-section::before { + counter-increment: docs-group; + content: counter(docs-group, decimal-leading-zero); + content: counter(docs-group, decimal-leading-zero) / ""; + margin-right: var(--sp-3); + letter-spacing: 0.04em; + font-variant-numeric: tabular-nums; + color: var(--accent); } -table { - border-collapse: collapse; - width: 100%; - font-size: 0.92rem; +.docs-menu > .docs-section:first-of-type, +.docs-menu > summary + .docs-section { + margin-top: var(--sp-2); } -th, -td { - text-align: left; - padding: 10px 14px; - border-bottom: 1px solid var(--line); - vertical-align: top; +.docs-nav ul { + list-style: none; + margin: 0; + padding: 0; } -th { - font-size: 0.78rem; - text-transform: uppercase; - letter-spacing: 0.05em; - color: var(--muted); - background: var(--bg-subtle); - font-weight: 700; +/* Rows of 28 px, so the whole list fits a laptop's screen. The page you are + on: the accent's wash, its ink and a bar at its edge. */ +.docs-nav li a { + position: relative; + display: block; + padding: 3px var(--sp-3); + line-height: 1.45; + border-radius: var(--r-2); + color: var(--ink-2); + text-decoration: none; } -tr:last-child td { - border-bottom: none; +/* The list scrolls, and would cut a focus ring drawn outside a link: the + ring is drawn just inside it instead. */ +.docs-menu a:focus-visible { + outline-offset: -3px; } -td.num, -th.num { - text-align: right; - font-variant-numeric: tabular-nums; +@media (hover: hover) { + .docs-nav li a:hover { + background: var(--surface-3); + color: var(--ink-1); + } } -.deep { - display: grid; - grid-template-columns: repeat(3, minmax(0, 1fr)); - gap: 12px; +.docs-nav li a[aria-current="page"] { + background: var(--accent-wash); + color: var(--accent); + font-weight: var(--fw-semibold); } -@media (max-width: 860px) { - .deep { - grid-template-columns: minmax(0, 1fr); +/* Where the list scrolls, the page you are on stays at its top while its + contents scroll under it, so a long page's sections never carry its name + out of sight. */ +@media (min-width: 861px) { + .docs-menu a[aria-current="page"] { + position: sticky; + top: 0; + z-index: 1; } } -.deep a { - display: block; - text-decoration: none; - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 10px; - padding: 14px 16px; - color: var(--ink); +.docs-nav li a[aria-current="page"]::before { + content: ""; + position: absolute; + left: 0; + top: 6px; + bottom: 6px; + width: 2px; + border-radius: 1px; + background: var(--accent); } -.deep a:hover { - border-color: var(--accent); +/* The page's contents, under its entry: a rail of hairline in line with the + page names, the section being read on an accent stretch of it, in full + ink. The h3s stay listed under their h2 all the time, so the list never + folds or unfolds, and nothing under it moves, as you read. On a phone, + where the list is folded at the top, the contents are the box under the + h1 instead. */ +.docs-nav .page-toc { + margin: var(--sp-1) 0 var(--sp-2) var(--sp-3); } -.deep a b { - display: block; - margin-bottom: 2px; +.docs-nav .page-toc a { + padding: 4px var(--sp-2) 4px var(--sp-3); + border-left: 1px solid var(--line-2); + border-radius: 0 var(--r-2) var(--r-2) 0; + color: var(--ink-3); + font-size: var(--fs-n2); + line-height: 1.4; } -.deep a span { - display: block; - font-size: 0.86rem; - line-height: 1.45; - color: var(--muted); +.docs-nav .page-toc .toc-3 a { + padding-left: calc(var(--sp-3) * 2); } -/* โ”€โ”€ Documentation pages โ”€โ”€ */ +/* A file name in the contents stays whole: a name wraps at its spaces, never + at a file name's hyphen. */ +.page-toc .nw, +.toc-inline .nw { + white-space: nowrap; +} -.docs-layout { - display: grid; - grid-template-columns: 230px minmax(0, 1fr) 210px; - gap: 44px; - padding-top: 32px; - padding-bottom: 72px; +@media (hover: hover) { + .docs-nav .page-toc a:hover { + color: var(--ink-1); + } } -.docs-home { - grid-template-columns: 230px minmax(0, 1fr); +/* Marked by its colour and the bar, not by weight, which would rewrap a long + name and move the list under it. */ +.docs-nav .page-toc a[aria-current="location"] { + color: var(--ink-1); } -@media (max-width: 1180px) { - .docs-layout { - grid-template-columns: 210px minmax(0, 1fr); - } +.docs-nav .page-toc a[aria-current="location"]::before { + content: ""; + position: absolute; + left: -1px; + top: 3px; + bottom: 3px; + width: 2px; + border-radius: 1px; + background: var(--accent); +} - .docs-toc { +/* The section an h3 being read is in: its name in the second ink. */ +.docs-nav .page-toc .in > a:not([aria-current]) { + color: var(--ink-2); +} + +@media (max-width: 860px) { + .page-toc { display: none; } } +/* The contents where there is no sidebar (a phone): folded under the h1, + its sections only. Like the box of pages above it, it takes the whole + column, so the two share their edges. */ +.toc-inline { + display: none; +} + @media (max-width: 860px) { - .docs-layout, - .docs-home { - grid-template-columns: minmax(0, 1fr); - gap: 18px; - padding-top: 18px; + .prose .toc-inline { + display: block; + margin: 0 0 var(--sp-7); + padding: 0 var(--sp-4); + border: 1px solid var(--line-2); + border-radius: var(--r-4); + background: var(--surface-1); + font-size: var(--fs-n1); } } -.docs-nav, -.docs-toc { - position: sticky; - top: 80px; - align-self: start; - font-size: 0.92rem; +.toc-inline summary { + cursor: pointer; + padding: var(--sp-3) 0; + font-weight: var(--fw-semibold); + color: var(--ink-1); } -/* A sticky box stacks on its own, so the sidebar sits above the text it - spreads its search results over. */ -.docs-nav { - z-index: 10; +/* The foldouts of a phone's docs page open with the questions' +, not the + browser's triangle. */ +.docs-menu summary, +.toc-inline summary { + list-style: none; + align-items: center; + justify-content: space-between; + gap: var(--sp-4); } -/* The page list and the contents scroll on their own; the search box does - not, so its results can spread over the page instead of being cut off. */ -.docs-menu, -.docs-toc { - max-height: calc(100vh - 150px); - overflow: auto; +.toc-inline summary { + display: flex; } -.docs-toc { - max-height: calc(100vh - 96px); +:is(.docs-menu, .toc-inline) summary::-webkit-details-marker { + display: none; } -@media (max-width: 860px) { - .docs-nav { - position: static; - } - - .docs-menu { - max-height: none; - } +.prose .toc-inline ul { + list-style: none; + margin: 0; + padding: 0 0 var(--sp-3); } -.docs-search { - position: relative; - margin-bottom: 16px; +.prose .toc-inline li { + margin: 0; } -.docs-search input { - width: 100%; - font: 0.92rem var(--font); - padding: 9px 12px; - border: 1px solid var(--line-strong); - border-radius: 9px; - background: var(--bg-elev); - color: var(--ink); +.toc-inline a { + display: block; + padding: var(--sp-1) 0 var(--sp-1) var(--sp-3); + border-left: 1px solid var(--line-2); + color: var(--ink-2); + text-decoration: none; + line-height: 1.4; } -.docs-results { - position: absolute; - z-index: 20; - left: 0; - right: -160px; - top: calc(100% + 6px); - background: var(--bg-elev); - border: 1px solid var(--line-strong); - border-radius: 10px; - box-shadow: var(--shadow); - padding: 6px; - max-height: 60vh; - overflow: auto; +.toc-inline a.active { + border-left-color: var(--accent); + color: var(--ink-1); } -@media (max-width: 860px) { - .docs-results { - right: 0; +@media (hover: hover) { + .toc-inline a:hover { + border-left-color: var(--accent); + color: var(--ink-1); } } -.docs-results a { - display: block; - padding: 8px 10px; - border-radius: 7px; - text-decoration: none; - color: var(--ink); +/* Long-form text: headings, bold and table heads in full ink, the rest a + step softer, so a page of text does not glare. */ + +.prose { + min-width: 0; + max-width: 820px; + color: var(--ink-2); + line-height: var(--lh-body); } -.docs-results a:hover, -.docs-results a:focus, -.docs-results a.active { - background: var(--bg-subtle); - outline: none; +.prose :is(h1, h2, h3, h4, h5, h6, strong, b, th) { + color: var(--ink-1); } -.docs-results b { - display: block; - font-size: 0.9rem; +/* A reading measure for running text; code, tables and drawings take the + whole column. */ +.prose > :is(p, ul, ol, blockquote), +.prose .callout { + max-width: 68ch; } -.docs-results small { - display: block; - color: var(--muted); - font-size: 0.8rem; - line-height: 1.35; - margin-top: 2px; +.prose .crumb-row { + display: flex; + flex-wrap: wrap; + justify-content: space-between; + align-items: baseline; + gap: 2px var(--sp-5); + margin-bottom: var(--sp-4); + font-size: var(--fs-n2); + color: var(--ink-3); +} + +.prose .crumb-row p { + margin: 0; } -.docs-results mark { - background: color-mix(in srgb, var(--accent) 30%, transparent); - color: inherit; - border-radius: 3px; +.prose .crumb-row a { + color: var(--ink-3); } -.docs-results .none { - padding: 8px 10px; - color: var(--muted); - font-size: 0.88rem; +@media (hover: hover) { + .prose .crumb-row a:hover { + color: var(--accent); + } } -.docs-menu summary { - display: none; +/* Where the page is: a caps label in the accent, as a chapter's kicker. */ +.prose .crumb { + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + line-height: var(--lh-label); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + color: var(--accent); } -@media (max-width: 860px) { - .docs-menu { - border: 1px solid var(--line); - border-radius: 10px; - background: var(--bg-elev); - padding: 4px 14px 8px; - } +.prose .crumb a { + color: var(--accent); + text-decoration: none; +} - .docs-menu summary { - display: list-item; - cursor: pointer; - font-weight: 650; - padding: 8px 0; +@media (hover: hover) { + .prose .crumb a:hover { + color: var(--accent-hover); + text-decoration: underline; } } -.docs-section { - font-size: 0.75rem; - font-weight: 700; - text-transform: uppercase; - letter-spacing: 0.06em; - color: var(--muted); - margin: 16px 0 6px; +.prose .ver { + font-size: var(--fs-n2); } -.docs-nav ul, -.docs-toc ul { - list-style: none; - margin: 0; - padding: 0; +.prose h1 { + font-size: var(--fs-title); + font-weight: var(--fw-display); + letter-spacing: var(--tr-display); + line-height: var(--lh-display); + margin-bottom: var(--sp-4); } -.docs-nav li a { - display: block; - padding: 5px 10px; - border-radius: 7px; - color: var(--ink-soft); - text-decoration: none; +/* What the page is for, in one line under its title. */ +.prose .standfirst { + max-width: 40em; + margin: calc(-1 * var(--sp-1)) 0 var(--sp-4); + font-size: var(--fs-1); + line-height: var(--lh-read); + color: var(--ink-2); + text-wrap: pretty; } -.docs-nav li a:hover { - background: var(--bg-subtle); - color: var(--ink); +@media (max-width: 600px) { + .prose .standfirst { + font-size: var(--fs-0); + } } -.docs-nav li a[aria-current="page"] { - background: var(--accent-soft); - color: var(--accent); - font-weight: 650; +.prose .doc-meta { + max-width: none; + margin-bottom: var(--sp-7); + padding-bottom: var(--sp-6); + border-bottom: 1px solid var(--line-1); + font-size: var(--fs-n2); + color: var(--ink-3); } -.docs-toc p { - font-size: 0.75rem; - font-weight: 700; - text-transform: uppercase; - letter-spacing: 0.06em; - color: var(--muted); - margin-bottom: 8px; +/* Docs headings: h2 a chapter (27px bold, under a hairline), h3 a section + (21.5px), h4 a run-in title (17px). */ +.prose h2 { + font-size: var(--fs-4); + font-weight: var(--fw-bold); + letter-spacing: var(--tr-heading); + line-height: var(--lh-heading); + margin: var(--sp-10) 0 var(--sp-4); } -.docs-toc li a { - display: block; - padding: 3px 0 3px 10px; - border-left: 2px solid var(--line); - color: var(--muted); - text-decoration: none; - line-height: 1.35; +.prose h3 { + font-size: var(--fs-2); + line-height: var(--lh-heading); + margin: var(--sp-8) 0 var(--sp-3); } -.docs-toc li.toc-3 a { - padding-left: 22px; - font-size: 0.86rem; +.prose h4, +.prose h5, +.prose h6 { + font-size: var(--fs-0); + letter-spacing: 0; + margin: var(--sp-7) 0 var(--sp-2); } -.docs-toc li a:hover { - color: var(--ink); +/* An old anchor of a heading, just before it; site.js moves the address on + to the heading's own. */ +.anchor-alias { + display: block; + height: 0; } -.docs-toc li a.active { - border-left-color: var(--accent); - color: var(--ink); +/* A heading and its # link: the link sits in the gutter to the left of the + heading, and on a narrow screen just after its text. The heading's margins + move to the pair, sized as the heading's. Anchored headings land under the + sticky header through html's scroll-padding-top alone. */ +.prose .hx { + position: relative; + display: flex; + align-items: baseline; + gap: 0.35em; } -/* Long-form text. */ - -.prose { - min-width: 0; - max-width: 780px; +.prose .hx-2 { + font-size: var(--fs-4); + margin: var(--sp-8) 0 var(--sp-4); + padding-top: var(--sp-5); + border-top: 1px solid var(--line-1); } -.prose .crumb { - font-size: 0.85rem; - color: var(--muted); - margin-bottom: 8px; +/* A chapter jumped to lands with its rule, not its words, where the page + clears the header: level with the sidebar's search box. */ +.prose .hx-2 > h2 { + scroll-margin-top: calc(var(--sp-5) + 1px); } -.prose .crumb a { - color: var(--muted); +.prose .hx-3 { + font-size: var(--fs-2); + margin: var(--sp-8) 0 var(--sp-3); } -.prose h1 { - font-size: clamp(2rem, 4vw, 2.6rem); - font-weight: 800; - letter-spacing: -0.03em; - margin-bottom: 18px; +.prose :is(.hx-4, .hx-5, .hx-6) { + font-size: var(--fs-0); + margin: var(--sp-7) 0 var(--sp-2); } -.prose h2 { - font-size: 1.5rem; - margin: 2.2em 0 0.7em; +/* A chapter that opens the page needs no rule over it: the head has one. */ +.prose .doc-meta + .hx-2, +.prose .doc-meta + .toc-inline + .hx-2 { + margin-top: 0; + padding-top: 0; + border-top: 0; } -.prose h3 { - font-size: 1.15rem; - margin: 1.8em 0 0.6em; +@media (min-width: 861px) { + .prose .hx-2 > .anchor { + top: var(--sp-5) !important; + } } -.prose h4, -.prose h5, -.prose h6 { - font-size: 1rem; - margin: 1.5em 0 0.5em; +.prose .hx > :is(h2, h3, h4, h5, h6) { + min-width: 0; + margin: 0; } .prose h1, .prose h2, .prose h3, .prose h4 { - scroll-margin-top: 80px; /* Headings that are file names have no spaces to break at. */ - overflow-wrap: anywhere; + overflow-wrap: break-word; } .prose .anchor { - margin-left: 8px; - color: var(--line-strong); + flex: none; + color: var(--muted); text-decoration: none; + font-weight: var(--fw-medium); + line-height: 1.15; opacity: 0; - font-weight: 500; } -.prose h2:hover .anchor, -.prose h3:hover .anchor, -.prose h4:hover .anchor, -.prose .anchor:focus { +.prose .anchor:focus-visible { opacity: 1; } +@media (hover: hover) { + .prose .hx:hover .anchor { + opacity: 1; + } +} + +@media (hover: hover) { + .prose .anchor:hover { + color: var(--accent); + } +} + +@media (min-width: 861px) { + .prose .hx .anchor { + position: absolute; + left: -1.1em; + top: 0; + } +} + +/* On a touch screen there is no hover to show it. */ +@media (hover: none) { + .prose .anchor { + opacity: 1; + } +} + +/* Narrower, the # follows the heading's last word, however the heading + wraps, rather than a column of its own at the right. */ +@media (max-width: 860px) { + .prose .hx { + display: block; + text-wrap: balance; + line-height: var(--lh-heading); + } + + .prose .hx > :is(h2, h3, h4, h5, h6) { + display: inline; + } + + .prose .hx > .anchor { + margin-left: 0.3em; + } +} + .prose p, .prose ul, .prose ol, @@ -2469,7 +6531,7 @@ th.num { .prose .tabset, .prose .fig, .prose .figure { - margin: 0 0 1.1em; + margin: 0 0 var(--sp-5); } .prose ul, @@ -2478,7 +6540,7 @@ th.num { } .prose li { - margin: 0.3em 0; + margin: var(--sp-1) 0; } .prose li > ul, @@ -2489,16 +6551,17 @@ th.num { .prose hr { border: 0; - border-top: 1px solid var(--line); - margin: 2em 0; + border-top: 1px solid var(--line-1); + margin: var(--sp-9) 0; } .prose blockquote { - border-left: 3px solid var(--accent); - background: var(--bg-subtle); - border-radius: 0 10px 10px 0; - padding: 12px 16px; - color: var(--ink-soft); + border: 1px solid var(--line-2); + box-shadow: inset 3px 0 0 var(--accent); + background: var(--surface-1); + border-radius: var(--r-3); + padding: var(--sp-4) var(--sp-5); + color: var(--ink-2); margin-left: 0; margin-right: 0; } @@ -2508,11 +6571,129 @@ th.num { } .prose .table-wrap { - font-size: 0.9rem; + font-size: var(--fs-n1); } -.prose td code { - white-space: normal; +/* The parameters page's tables, one under the other, start their second + column at one place: the first holds the longest option name. */ +@media (min-width: 861px) { + .prose table.params th:first-child { + width: 16rem; + } +} + +/* A type is a word to read, not a chip of code. */ +.prose td.type { + color: var(--ink-3); + font-size: var(--fs-n2); + white-space: nowrap; +} + +/* An option's row, opened from a link to it. */ +.prose tr:target > td { + background: var(--accent-wash); +} + +.prose tr:target > td:first-child { + box-shadow: inset 3px 0 0 var(--accent); +} + +/* A chip naming an option links to its row of the parameters page. */ +.prose a.opt-link { + text-decoration: none; +} + +.prose a.opt-link code { + color: var(--accent); + border-color: color-mix(in srgb, var(--accent) 35%, var(--line)); +} + +@media (hover: hover) { + .prose a.opt-link:hover code { + border-color: var(--accent); + color: var(--accent-hover); + } +} + +@media (max-width: 600px) { + .prose th, + .prose td { + padding: 10px; + } + + /* A table of four columns or more becomes one card per row: the first + cell as its title, the second as its text, the rest on one muted line, + each with its column's name. */ + .prose .table-wrap.cards-wrap { + overflow: visible; + border: 0; + border-radius: 0; + background: none; + } + + .prose table.cards, + .prose table.cards tbody, + .prose table.cards tr, + .prose table.cards td { + display: block; + } + + .prose table.cards thead { + position: absolute; + width: 1px; + height: 1px; + overflow: hidden; + clip-path: inset(50%); + white-space: nowrap; + } + + .prose table.cards tr { + margin-bottom: var(--sp-3); + padding: var(--sp-3) var(--sp-4); + border: 1px solid var(--line-2); + border-radius: var(--r-3); + background: var(--surface-1); + } + + .prose table.cards tr:target { + background: var(--accent-soft); + border-color: var(--accent); + } + + .prose table.cards td { + padding: 0; + border: 0; + } + + .prose table.cards td:first-child { + margin-bottom: 4px; + font-weight: var(--fw-semibold); + color: var(--ink); + } + + .prose table.cards tr:target > td:first-child { + box-shadow: none; + } + + .prose table.cards td:nth-child(2) { + margin-bottom: 6px; + } + + .prose table.cards td:nth-child(n + 3) { + display: inline; + margin-right: 14px; + font-size: var(--fs-n2); + color: var(--muted); + } + + .prose table.cards td:nth-child(n + 3):empty { + display: none; + } + + .prose table.cards td:nth-child(n + 3)::before { + content: attr(data-label) ": "; + font-weight: var(--fw-semibold); + } } .prose .figure img { @@ -2532,9 +6713,20 @@ th.num { display: none; } +.prose .to-top { + margin-top: var(--sp-7); + font-size: var(--fs-n1); +} + .prose .edit { - margin-top: 2.5em; - font-size: 0.9rem; + margin-top: var(--sp-8); + font-size: var(--fs-n1); +} + +/* Under "Back to the top", the pair reads as one, with room between them + for each one's finger target (see Touch and pointer). */ +.prose .to-top + .edit { + margin-top: var(--sp-6); } .prose .edit a { @@ -2548,13 +6740,22 @@ th.num { height: 16px; } -/* GitHub's alerts (> [!WARNING]) as callouts. */ +/* Where the headings' # hangs in the gutter, so does the mark, and the + words line up with "Back to the top". */ +@media (min-width: 861px) { + .prose .edit a { + margin-left: calc(-16px - var(--sp-2)); + } +} + +/* GitHub's alerts (> [!WARNING]) as callouts, which say their kind in words + as well as in colour. */ .callout { - border: 1px solid var(--line); - border-left: 4px solid var(--c); - background: color-mix(in srgb, var(--c) 8%, var(--bg-elev)); - border-radius: 0 10px 10px 0; - padding: 12px 16px 2px; + border: 1px solid color-mix(in srgb, var(--c) 32%, var(--line-2)); + box-shadow: inset 3px 0 0 var(--c); + background: color-mix(in srgb, var(--c) 6%, var(--surface-1)); + border-radius: var(--r-3); + padding: var(--sp-4) var(--sp-5) var(--sp-1); --c: var(--info); } @@ -2575,84 +6776,351 @@ th.num { } .callout-title { - font-weight: 700; - color: var(--ink); - margin-bottom: 6px !important; + font-weight: var(--fw-semibold); + color: var(--ink-1); + margin-bottom: var(--sp-2) !important; +} + +.callout-kind { + display: block; + margin-bottom: var(--sp-1); + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + color: var(--c); +} + +.callout > p:last-child, +.callout > .code:last-child { + margin-bottom: var(--sp-3); +} + +/* A block with no head row shows Copy when pointed at; with a head row, or + on a touch screen, it is always there. */ +@media (hover: hover) and (pointer: fine) { + .prose .code.bare .copy { + opacity: 0; + } + + .prose .code.bare:hover .copy, + .prose .code .copy:focus-visible, + .prose .code .copy.done { + opacity: 1; + } +} + +@media (hover: none) { + .prose .code .copy { + opacity: 1; + } +} + +.tabset .tabs { + margin-bottom: 8px; +} + +/* The Docs tab: the questions people come with, before the list of pages. */ +.prose .hub-tasks { + margin: var(--sp-5) 0 var(--sp-2); +} + +.prose .hub-label { + margin-bottom: var(--sp-3); + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + color: var(--ink-3); +} + +.prose .hub-tasks ul { + display: flex; + flex-wrap: wrap; + gap: 8px; + margin: 0; + padding: 0; + list-style: none; +} + +.prose .hub-tasks li { + margin: 0; +} + +.prose .hub-tasks a { + display: inline-block; + padding: 7px var(--sp-4); + border: 1px solid var(--line-2); + border-radius: var(--r-pill); + background: var(--control); + color: var(--ink-1); + font-size: var(--fs-n1); + font-weight: var(--fw-semibold); + text-decoration: none; +} + +@media (hover: hover) { + .prose .hub-tasks a:hover { + border-color: var(--accent-line); + background: var(--accent-wash); + color: var(--accent); + } +} + +/* On paper: light colours, the text alone, and the address of every link + that leaves the site. */ +/* On paper, and in a high-contrast theme, the graticule, its ruler and the + plates' corner marks step out: they are drawn for a screen, and a forced + palette would turn them into lines of text colour. The rail of chapters + and the bar that names the section being read step out on paper only; in + forced colours the rail stays, its chapter marked in the system's + highlight. */ +@media print { + body { + background-image: none; + } + + .top::after, + .rail, + .here-bar, + :is(.demo, .cases, .case, .metro-wrap, .how-stage-card, .excerpt, .report-frame, .lost)::after { + display: none !important; + } +} + +@media (forced-colors: active) { + body { + background-image: none; + } + + .top::after, + :is(.demo, .cases, .case, .metro-wrap, .how-stage-card, .excerpt, .report-frame, .lost)::after { + display: none; + } } -.callout > p:last-child, -.callout > .code:last-child { - margin-bottom: 12px; -} +@media print { + :root, + :root[data-theme], + .docs { + --surface-sunk: #f5f7f6; + --surface-0: #ffffff; + --surface-1: #ffffff; + --surface-2: #ffffff; + --surface-3: #f3f5f4; + --ink-1: #111816; + --ink-2: #242e2b; + --ink-3: #4f5b58; + --line-1: #d5dcda; + --line-2: #b9c3c0; + --line-3: #7c8985; + --accent-wash: #e2f3ef; + --accent-line: #8cc5ba; + --control: #ffffff; + --halo: #ffffff; + --shadow-2: none; + --bg: #ffffff; + --bg-elev: #ffffff; + --bg-subtle: #f3f5f4; + --ink: #111816; + --ink-soft: #242e2b; + --muted: #4f5b58; + --line: #d5dcda; + --line-strong: #b9c3c0; + --accent: #055c4d; + --accent-soft: #e2f3ef; + --code-bg: #f5f7f6; + --code-border: #d5dcda; + --code-ink: #111816; + --code-muted: #4f5b58; + --code-flag: #055c4d; + --code-value: #7a3f00; + --on-accent: #ffffff; + --glow: none; + --brand-teal: #155b4e; + --brand-aqua: #11978b; + --brand-slate: #4f7682; + /* The drawings in the light theme's inks, which hold on white. */ + --bakta: #a85a06; + --ref: #5845cc; + --amap: #07796a; + --drop: #7d8986; + --on-track: #ffffff; + --pseudo: #c0244a; + --ok: #0c724d; + --note: #b7791f; + --warn: #9b5806; + --danger: #c0244a; + --info: #1f68c4; + --m-main: #0b8a75; + --m-ref: #5845cc; + --m-db: #b5650d; + --m-dup: #1f68c4; + --m-pan: #a6338f; + color-scheme: light; + background: #ffffff; + color: var(--ink); + } + + body { + background: #ffffff; + } + + /* What is for a screen leaves the paper: the header and footer, the ways + round a page, and the controls of the drawings (a drawing prints as it + ends, every layer in). */ + :is(.top, .docs-nav, .toc-inline, .copy, .docs-pager, .edit, .to-top, .foot, .skip, .crumb-row, .anchor, .demo-ctrl, .hero-segs, .how-pick, .how-steps, .how-nav, .builder-qs, .metro-opts, .metro-hint, .cases-hint) { + display: none !important; + } + + /* Paper shows no frames: every change is at its end at once. */ + *, + *::before, + *::after { + transition: none !important; + animation: none !important; + } + + /* The headline's second sentence, in the accent: a gradient clipped to + text does not print. */ + .grad { + background: none; + color: var(--accent); + -webkit-text-fill-color: currentColor; + } + + .code pre { + white-space: pre-wrap; + overflow-wrap: anywhere; + } + + :is(.demo, .case, .how-stage-card, .metro-wrap, .excerpt) { + break-inside: avoid; + box-shadow: none; + } + + .wrap.docs-layout { + display: block; + max-width: none; + padding: 0; + } + + .prose, + .prose > :is(p, ul, ol, blockquote), + .prose .callout { + max-width: none; + } + + .prose .code pre, + .prose .code:not(.cmd) pre { + white-space: pre-wrap; + overflow-wrap: anywhere; + overflow: visible; + background: none; + padding-right: 14px; + } + + .table-wrap { + overflow: visible; + background: none; + } -/* Commands wrap in a column of text rather than run under the copy button; - files and trees keep their layout and show the button on hover. */ -.prose .code[data-lang="bash"] pre, -.prose .code[data-lang="sh"] pre, -.prose .code[data-lang="console"] pre { - white-space: pre-wrap; - overflow-wrap: anywhere; -} + .prose :is(.code, .fig, figure, .callout, tr, .tabset) { + break-inside: avoid; + } -.prose .code:not([data-lang="bash"]):not([data-lang="sh"]):not([data-lang="console"]) .copy { - opacity: 0; - transition: opacity 0.15s ease; -} + .prose :is(.hx, h2, h3, h4) { + break-after: avoid; + } -.prose .code:hover .copy, -.prose .code .copy:focus, -.prose .code .copy.done { - opacity: 1 !important; + .prose a[href^="http"]::after { + content: " (" attr(href) ")"; + font-size: 0.85em; + color: var(--muted); + overflow-wrap: anywhere; + } } -.tabset .tabs { - margin-bottom: 8px; +/* A drawing in a page of text, with the room of a block round it; right + under its heading it needs none above. */ +.prose .fig { + margin: var(--sp-6) 0; } -/* A drawing in a page of text. */ -.prose .fig { - margin-top: 1.4em; - margin-bottom: 1.6em; +.prose .hx + .fig { + margin-top: 0; } .prose .fig.case { - padding: 16px 16px 12px; + padding: var(--sp-4); } +/* A figure's caption, as a paper sets one: "Figure n" in caps, counted down + the page (drawn, not read), then its first sentence in full ink (set bold + by the docs builder), then the rest. */ .prose figcaption { - font-size: 0.86rem; - color: var(--muted); - margin-top: 6px; + font-size: var(--fs-n2); + line-height: var(--lh-ui); + color: var(--ink-3); + margin-top: var(--sp-3); + text-wrap: pretty; +} + +.prose figcaption::before { + counter-increment: figure; + content: "Figure " counter(figure); + content: "Figure " counter(figure) / ""; + margin-right: var(--sp-2); + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + color: var(--accent); +} + +.cap-lead { + color: var(--ink-1); + font-weight: var(--fw-semibold); } .pager { display: grid; - grid-template-columns: 1fr 1fr; - gap: 14px; - margin-top: 18px; + grid-template-columns: repeat(2, minmax(0, 1fr)); + gap: var(--sp-4); + margin-top: var(--sp-6); } .pager a { - border: 1px solid var(--line); - border-radius: 10px; - background: var(--bg-elev); - padding: 12px 16px; + min-width: 0; + overflow-wrap: break-word; + border: 1px solid var(--line-2); + border-radius: var(--r-4); + background: var(--surface-1); + padding: var(--sp-4) var(--sp-5); text-decoration: none; - color: var(--ink); - font-weight: 650; -} - -.pager a:hover { - border-color: var(--accent); + color: var(--ink-1); + font-size: var(--fs-1); + font-weight: var(--fw-semibold); + letter-spacing: var(--tr-heading); + transition: color var(--dur-state) var(--ease), border-color var(--dur-state) var(--ease), background-color var(--dur-state) var(--ease); +} + +@media (hover: hover) { + .pager a:hover { + border-color: var(--accent-line); + background: var(--surface-2); + color: var(--accent); + } } .pager a span { display: block; - font-size: 0.78rem; - font-weight: 600; - color: var(--muted); + margin-bottom: var(--sp-1); + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + color: var(--ink-3); text-transform: uppercase; - letter-spacing: 0.05em; + letter-spacing: var(--tr-caps); } .pager .next { @@ -2660,9 +7128,40 @@ th.num { text-align: right; } +/* On a phone the two cards stack, so a long title keeps its words whole. */ +@media (max-width: 480px) { + .pager { + grid-template-columns: minmax(0, 1fr); + } + + .pager .next { + grid-column: 1; + } +} + +/* The Docs tab's groups, numbered as the sidebar numbers them. */ +.docs-home .prose { + counter-reset: docs-group; +} + .docs-group { - font-size: 1.1rem !important; - margin: 1.6em 0 0.7em !important; + font-size: var(--fs-0) !important; + margin: var(--sp-7) 0 var(--sp-3) !important; + display: flex; + align-items: baseline; + gap: var(--sp-3); +} + +.docs-group::before { + counter-increment: docs-group; + content: counter(docs-group, decimal-leading-zero); + content: counter(docs-group, decimal-leading-zero) / ""; + font-weight: var(--fw-light); + font-size: var(--fs-2); + line-height: 1; + letter-spacing: var(--tr-numeral); + font-variant-numeric: tabular-nums; + color: var(--accent); } /* โ”€โ”€ Figures written out by the docs builder โ”€โ”€ */ @@ -2670,15 +7169,15 @@ th.num { .fig.case-bars, .fig.results-map, .fig.prepare-ref { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; - padding: 16px; + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); + padding: var(--sp-4); } .cb-rows { display: grid; - gap: 14px; + gap: var(--sp-4); } .cb-row { @@ -2688,12 +7187,12 @@ th.num { } .cb-name { - font-weight: 650; - font-size: 0.9rem; + font-weight: var(--fw-semibold); + font-size: var(--fs-n1); } .cb-total { - font-size: 0.82rem; + font-size: var(--fs-n2); color: var(--muted); font-variant-numeric: tabular-nums; } @@ -2707,7 +7206,7 @@ th.num { .cb-seg { position: relative; - border-radius: 4px; + border-radius: var(--r-1); min-width: 3px; display: flex; align-items: center; @@ -2716,7 +7215,7 @@ th.num { } .cb-n { - font: 650 0.74rem var(--mono); + font: var(--fw-semibold) var(--fs-n3) var(--mono); color: var(--on-track); white-space: nowrap; padding: 0 4px; @@ -2727,6 +7226,30 @@ th.num { text-overflow: clip; } +/* On a phone a number takes less room round it, and one its segment cannot + hold whole is not shown cut: the key under the bar gives every count. */ +@media (max-width: 600px) { + .cb-n { + padding: 0 2px; + } +} + +.cb-seg { + container-type: inline-size; +} + +@container (max-width: 25px) { + .cb-n { + visibility: hidden; + } +} + +@container (max-width: 39px) { + .cb-n.cb-n-wide { + visibility: hidden; + } +} + .cb-agree, .cb-key.cb-agree { background: var(--ok); @@ -2775,8 +7298,8 @@ th.num { grid-column: 1 / -1; display: flex; flex-wrap: wrap; - gap: 4px 14px; - font-size: 0.8rem; + gap: var(--sp-1) var(--sp-4); + font-size: var(--fs-n2); color: var(--muted); } @@ -2796,9 +7319,9 @@ th.num { grid-template-columns: 14px minmax(150px, max-content) minmax(0, 1fr); gap: 4px 12px; align-items: baseline; - padding: 9px 0; + padding: var(--sp-2) 0; border-bottom: 1px solid var(--line); - font-size: 0.88rem; + font-size: var(--fs-n2); } .rm-row:last-of-type { @@ -2809,7 +7332,8 @@ th.num { width: 14px; height: 6px; border-radius: 3px; - align-self: center; + align-self: start; + margin-top: var(--sp-2); } .rm-main { @@ -2829,7 +7353,7 @@ th.num { } .rm-folder { - font-weight: 650; + font-weight: var(--fw-semibold); justify-self: start; } @@ -2839,13 +7363,13 @@ th.num { } .rm-by { - font: 0.8rem var(--mono); + font: var(--fs-n2)/var(--lh-ui) var(--mono); color: var(--ink-soft); } .rm-when { color: var(--muted); - font-size: 0.84rem; + font-size: var(--fs-n2); } .rm-when code { @@ -2864,7 +7388,7 @@ th.num { .pr-in, .pr-out { - font-size: 0.86rem; + font-size: var(--fs-n2); color: var(--ink-soft); text-align: center; } @@ -2886,7 +7410,7 @@ th.num { .pr-step { background: var(--bg); border: 1px solid var(--line); - border-radius: 12px; + border-radius: var(--r-3); padding: 12px; display: flex; flex-direction: column; @@ -2894,11 +7418,11 @@ th.num { } .pr-step b { - font-size: 0.95rem; + font-size: var(--fs-n1); } .pr-step p { - font-size: 0.85rem !important; + font-size: var(--fs-n2) !important; color: var(--muted); margin: 0 !important; } @@ -2911,24 +7435,54 @@ th.num { border-radius: 50%; background: var(--m-ref); color: var(--on-track); - font-weight: 750; - font-size: 0.8rem; + font-weight: var(--fw-bold); + font-size: var(--fs-n2); } .pr-count { margin-top: auto; - font: 650 0.85rem var(--mono); + font: var(--fw-semibold) var(--fs-n2) var(--mono); color: var(--ref); } /* โ”€โ”€ How it works: one story through seven steps โ”€โ”€ */ .how-hero { - padding-bottom: 20px; + padding-top: var(--sp-9); + padding-bottom: var(--sp-5); +} + +@media (max-width: 600px) { + .how-hero { + padding-top: var(--sp-7); + } +} + +/* The crumbs as the docs set them: a caps label in the accent. */ +.how-hero .crumb { + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + line-height: var(--lh-label); + letter-spacing: var(--tr-caps); + text-transform: uppercase; + color: var(--accent); + margin-bottom: var(--sp-4); +} + +.how-hero .crumb a { + color: var(--accent); + text-decoration: none; +} + +@media (hover: hover) { + .how-hero .crumb a:hover { + color: var(--accent-hover); + text-decoration: underline; + } } .how { - padding: 8px 0 40px; + padding: var(--sp-2) 0 var(--sp-12); } .how:focus { @@ -2937,42 +7491,137 @@ th.num { .how-pick { display: grid; - gap: 12px; - margin-bottom: 22px; + grid-template-columns: minmax(0, 1fr); + gap: var(--sp-6); + margin-bottom: var(--sp-6); } .how-group .q-title { - font-size: 0.8rem; + font-size: var(--fs-n3); + font-weight: var(--fw-bold); text-transform: uppercase; - letter-spacing: 0.06em; - color: var(--muted); - margin-bottom: 6px; + letter-spacing: var(--tr-caps); + color: var(--ink-3); + margin: 0 0 var(--sp-2); } -.how-group .seg button { +/* The stories are links (#key), so each can be opened, copied or shared. */ +.how-group .seg a { display: inline-flex; align-items: center; gap: 8px; + min-height: 36px; + border: 1px solid var(--line-2); + background: var(--control); + color: var(--ink-1); + font: var(--fw-semibold) var(--fs-n2) var(--font); + padding: 0 var(--sp-3) 0 var(--sp-4); + border-radius: var(--r-3); + text-decoration: none; +} + +@media (hover: hover) { + .how-group .seg a:hover { + border-color: var(--line-3); + background: color-mix(in srgb, var(--ink-1) 7%, var(--control)); + color: var(--ink-1); + } +} + +/* A gene, a station or a link of a drawing that the keyboard reaches is + scrolled clear of the sticky header: Chrome ignores the page's + scroll-padding for an SVG element. */ +.trk .hot, +.metro .stn, +.net .hot { + scroll-margin-top: var(--clear); + scroll-margin-bottom: 16px; +} + +/* A chip focused in a row that scrolls clears the row's faded edge. */ +.how-group .seg a { + scroll-margin-inline: 32px; +} + +.how-group .seg a[aria-current="true"] { + background: var(--accent); + border-color: var(--accent); + color: var(--on-accent); +} + +@media (hover: hover) { + .how-group .seg a[aria-current="true"]:hover, + .seg button:is([aria-pressed="true"], [aria-checked="true"]):hover, + .tab[aria-selected="true"]:hover { + background: var(--accent-hover); + border-color: var(--accent-hover); + color: var(--on-accent); + } } .how-tag { - font: 650 0.68rem var(--font); - letter-spacing: 0.04em; + font: var(--fw-bold) var(--fs-n3)/1 var(--font); + letter-spacing: var(--tr-caps); text-transform: uppercase; - color: var(--muted); + color: var(--ink-3); } -.seg button[aria-pressed="true"] .how-tag { +.seg a[aria-current="true"] .how-tag { color: inherit; } +/* On a phone each group is one row of chips that slides sideways, so the + drawing still starts on the first screen. */ +@media (max-width: 600px) { + .how-pick { + gap: 8px; + margin-bottom: 16px; + } + + .how-group .seg { + flex-wrap: nowrap; + overflow-x: auto; + scroll-snap-type: x proximity; + scroll-padding: 0 16px; + /* Room inside the scroller for a focused chip's ring, which it would + otherwise clip. */ + margin: -5px -16px -1px; + /* The end of the row clears the right fade, so the last chip, scrolled + to, shows whole. */ + padding: 5px 32px 5px 16px; + scrollbar-width: none; + /* Both edges fade, so a chip cut at either reads as more to swipe to; + the left fade lies over the row's own padding until it scrolls. */ + -webkit-mask-image: linear-gradient(90deg, transparent, #000 16px, #000 calc(100% - 28px), transparent); + mask-image: linear-gradient(90deg, transparent, #000 16px, #000 calc(100% - 28px), transparent); + } + + .how-group .seg::-webkit-scrollbar { + display: none; + } + + .how-group .seg a { + flex: none; + scroll-snap-align: start; + } +} + .how-layout { display: grid; - grid-template-columns: 250px minmax(0, 1fr); - gap: 22px; + grid-template-columns: 248px minmax(0, 1fr); + gap: var(--sp-7); align-items: start; } +/* From 1216px the stepper takes the front page's margin column, so the + stage starts where the front page's and the docs' text does. */ +@media (min-width: 1216px) { + .how-layout { + grid-template-columns: var(--col-margin) minmax(0, 1fr); + column-gap: var(--col-gap); + } +} + @media (max-width: 900px) { .how-layout { grid-template-columns: minmax(0, 1fr); @@ -2984,26 +7633,16 @@ th.num { margin: 0; padding: 0; display: grid; - gap: 4px; + gap: var(--sp-1); position: sticky; - top: 80px; + top: var(--clear); } @media (max-width: 900px) { .how-steps { position: static; grid-template-columns: repeat(7, minmax(0, 1fr)); - gap: 4px; - } - - .how-steps button { - justify-content: center; - padding: 8px 4px !important; - font-size: 0 !important; - } - - .how-steps .how-n { - font-size: 0.85rem; + gap: var(--sp-1); } } @@ -3015,22 +7654,43 @@ th.num { text-align: left; border: 1px solid transparent; background: none; - color: var(--ink-soft); - font: 600 0.92rem var(--font); - padding: 9px 10px; - border-radius: 10px; + color: var(--ink-2); + font: var(--fw-semibold) var(--fs-n1)/var(--lh-label) var(--font); + padding: var(--sp-2) var(--sp-3); + min-height: 44px; + border-radius: var(--r-3); cursor: pointer; } -.how-steps button:hover { - background: var(--bg-subtle); - color: var(--ink); +/* Pointed at, a step is outlined: quieter than the current one's raised + fill, so it never looks the more chosen of the two. */ +@media (hover: hover) { + .how-steps button:hover { + border-color: var(--line-1); + color: var(--ink-1); + } +} + +/* On a narrow screen the steps are a row of their numbers alone, each + centred in its button: the hidden name leaves no gap beside it. */ +@media (max-width: 900px) { + .how-steps button { + justify-content: center; + gap: 0; + padding: var(--sp-2) var(--sp-1); + font-size: 0; + } + + .how-steps .how-n { + font-size: var(--fs-n2); + } } .how-steps button[aria-current="step"] { - background: var(--bg-elev); - border-color: var(--line-strong); - color: var(--ink); + background: var(--surface-2); + border-color: var(--line-2); + box-shadow: var(--shadow-2); + color: var(--ink-1); } .how-n { @@ -3039,9 +7699,10 @@ th.num { width: 26px; height: 26px; border-radius: 50%; - border: 1.5px solid var(--line-strong); - font-size: 0.8rem; - font-weight: 750; + border: 1.5px solid var(--line-3); + font-size: var(--fs-n2); + font-weight: var(--fw-bold); + font-variant-numeric: tabular-nums; flex: none; } @@ -3054,46 +7715,101 @@ th.num { background: var(--accent); border-color: var(--accent); color: var(--on-accent); + box-shadow: var(--glow); } -.how-stage-card { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 16px; +/* A step that changes nothing in this story: hollow, dashed and quieter. */ +.how-steps button.same { + color: var(--ink-3); +} + +.how-steps button.same .how-n { + border-style: dashed; +} + +.how-steps button.same[aria-current="step"] .how-n { + background: transparent; + color: var(--accent); +} + +.how-same { + position: absolute; + width: 1px; + height: 1px; overflow: hidden; + clip-path: inset(50%); + white-space: nowrap; +} + +.how-stage-card { + position: relative; + background: var(--surface-2); + border: 1px solid var(--line-2); + border-radius: var(--r-5); + box-shadow: var(--shadow-2); min-width: 0; } +/* The card does not clip, so a tip can rise above the drawing. The bar + lies in a box as tall as the card's corner and clipped to its curve, so + the 3px stripe at its top bends exactly as the card's edge does. */ .how-progress { + position: absolute; + inset: 0 0 auto; + height: var(--r-5); + border-radius: calc(var(--r-5) - 1px) calc(var(--r-5) - 1px) 0 0; + overflow: hidden; + pointer-events: none; +} + +.how-progress::before { + content: ""; + position: absolute; + inset: 0 0 auto; height: 3px; - background: var(--line); + background: var(--line-1); } +/* While the story plays, the bar runs on to the next step in the time the + step is given (set by how.js). */ .how-progress i { + position: relative; display: block; - height: 100%; + height: 3px; width: 0; - background: linear-gradient(90deg, var(--brand-aqua), var(--brand-teal)); + background: var(--accent); transition: width 0.35s ease; } .how-genome { - margin: 14px 18px 0; - font-size: 0.82rem; - font-weight: 650; - color: var(--muted); + margin: calc(var(--sp-5) + 3px) var(--sp-6) 0; + min-height: 1.3em; + font-size: var(--fs-n2); + font-weight: var(--fw-semibold); + color: var(--ink-3); } +/* Room for the first story's drawing before the script draws it. */ .how-stage { - padding: 6px 18px 0; + padding: var(--sp-2) var(--sp-6) 0; display: grid; - gap: 8px; + gap: var(--sp-2); +} + +.how-stage:empty { + min-height: 158px; +} + +@media (hover: none), (max-width: 640px) { + .how-stage:empty { + min-height: 246px; + } } .how-panel-title { margin: 6px 0 0; - font-size: 0.8rem; - font-weight: 700; + font-size: var(--fs-n2); + font-weight: var(--fw-bold); color: var(--ink-soft); } @@ -3101,67 +7817,188 @@ th.num { position: relative; } -.how-legend { - padding: 0 18px; - margin-top: 4px; +/* A story drawn in two panels says "Tap a gene" once, under the last: the + first panel's line appears when one of its genes is tapped. */ +.how-panel:not(:last-child) .fig-readout:has(> .say-hint) { + display: none; } -.step-off { - opacity: 0 !important; - pointer-events: none; +.how-legend { + padding: 0 var(--sp-6); + margin-top: var(--sp-2); } +/* An element waits for its step hidden: it fades, then leaves the pointer + and the keyboard. */ .trk [data-from] { - transition: opacity 0.4s ease; + transition: opacity var(--dur-reveal) var(--ease), visibility 0s linear 0s; +} + +.trk .step-off { + opacity: 0; + visibility: hidden; + transition: opacity var(--dur-reveal) var(--ease), visibility 0s linear var(--dur-reveal); +} + +/* The name of a row still to come, quieter, and still readable: the same + treatment as the front page's rows to come. */ +.trk .ghost .row-label { + fill: var(--ink-3); + font-weight: var(--fw-medium); +} + +.trk .ghost :is(.row-line, .row-lead) { + opacity: 0.35; +} + +/* Back and Next sit right under the drawing, where the text under them + cannot move them. */ +.how-nav { + display: flex; + align-items: center; + gap: var(--sp-3); + justify-content: flex-end; + margin-top: var(--sp-4); + padding: var(--sp-4) var(--sp-6); + border-top: 1px solid var(--line-1); +} + +/* A button that cannot be pressed yet: no fill, a hairline edge, its words + in the third ink (still 4.5:1 and more, unlike a faded button). */ +.how-nav .btn[aria-disabled="true"] { + background: transparent; + border-color: var(--line-1); + color: var(--ink-3); + box-shadow: none; + cursor: not-allowed; +} + +.how-hint { + margin-right: auto; + font-size: var(--fs-n2); + color: var(--ink-3); + visibility: hidden; +} + +.tip-keys { + display: block; + margin-top: var(--sp-1); + font-size: var(--fs-n3); + color: var(--ink-3); +} + +:is(.how-hint, .tip-keys) kbd { + display: inline-block; + min-width: 1.6em; + padding: 0 4px; + border: 1px solid var(--line-strong); + border-bottom-width: 2px; + border-radius: var(--r-1); + font-size: var(--fs-n3); + text-align: center; + color: var(--ink-soft); +} + +.how-hint .k-genes { + display: none; +} + +.how[data-keys] .how-hint { + visibility: visible; +} + +.how[data-keys="genes"] .how-hint .k-steps { + display: none; +} + +.how[data-keys="genes"] .how-hint .k-genes { + display: inline; +} + +html:not(.js) .how-nav { + display: none; +} + +@media (hover: none) { + .how-hint { + display: none; + } +} + +/* Play and Next hold both of their words in one cell and show one, so each + is as wide as its longer word and a change of word moves nothing. */ +.how-nav .btn { + display: inline-grid; + justify-items: center; + white-space: nowrap; +} + +.how-nav .btn > span { + grid-area: 1 / 1; +} + +.how-nav .btn > .say-2, +.how-nav .btn.said-2 > .say-1 { + visibility: hidden; +} + +.how-nav .btn.said-2 > .say-2 { + visibility: visible; +} + +@media (max-width: 600px) { + .how-nav { + gap: 8px; + } + + .how-nav .btn { + flex: 1 1 auto; + padding-inline: 12px; + } +} + +@media (max-width: 359px) { + .how-nav .btn { + white-space: normal; + } } .how-text { - padding: 16px 18px 4px; - border-top: 1px solid var(--line); - margin-top: 12px; + padding: var(--sp-5) var(--sp-6) var(--sp-3); } .how-kicker { - font-size: 0.75rem; - font-weight: 750; + font-size: var(--fs-n3); + font-weight: var(--fw-bold); + line-height: var(--lh-label); text-transform: uppercase; - letter-spacing: 0.06em; + letter-spacing: var(--tr-caps); color: var(--accent); } .how-text h2 { - font-size: 1.35rem; - margin: 6px 0 8px; + font-size: var(--fs-3); + font-weight: var(--fw-bold); + letter-spacing: var(--tr-heading); + line-height: var(--lh-heading); + margin: var(--sp-2) 0 var(--sp-3); } .how-text p[data-step-text] { - color: var(--ink-soft); + color: var(--ink-2); max-width: 60em; - min-height: 4.6em; } .how-out pre { - margin: 12px 0 4px; - padding: 12px 14px; - border-radius: 10px; + margin: var(--sp-3) 0 var(--sp-1); + padding: var(--sp-3) var(--sp-4); + border-radius: var(--r-3); background: var(--code-bg); border: 1px solid var(--code-border); color: var(--code-ink); - font: 0.78rem/1.6 var(--mono); + font: var(--fs-n2)/1.6 var(--mono); white-space: pre-wrap; - overflow-wrap: anywhere; -} - -.how-nav { - display: flex; - gap: 10px; - justify-content: flex-end; - padding: 12px 18px 16px; -} - -.how-nav .btn:disabled { - opacity: 0.45; - cursor: default; + overflow-wrap: break-word; } .how-note { @@ -3172,29 +8009,29 @@ th.num { /* โ”€โ”€ The container: what you install, and what the image brings โ”€โ”€ */ .fig.image-fig { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); padding: 16px; } .img-grid { display: grid; grid-template-columns: minmax(0, 1fr) auto minmax(0, 1.25fr); - gap: 14px; + gap: var(--sp-4); align-items: center; } .img-host, .img-box { - border-radius: 12px; - padding: 12px 14px; + border-radius: var(--r-3); + padding: var(--sp-3) var(--sp-4); min-width: 0; } .img-host { border: 1px dashed var(--line-strong); - font-size: 0.86rem; + font-size: var(--fs-n2); } .prose .img-host ul { @@ -3213,10 +8050,10 @@ th.num { .prose .img-title { margin: 0 0 8px; - font-size: 0.76rem; - font-weight: 750; + font-size: var(--fs-n3); + font-weight: var(--fw-bold); text-transform: uppercase; - letter-spacing: 0.06em; + letter-spacing: var(--tr-caps); color: var(--muted); display: flex; justify-content: space-between; @@ -3228,14 +8065,14 @@ th.num { } .img-box .img-title code { - font-size: 0.86rem; + font-size: var(--fs-n2); text-transform: none; letter-spacing: 0; color: var(--ink); } .img-box .img-title span { - font: 600 0.72rem/1.4 var(--mono); + font: var(--fw-semibold) var(--fs-n3)/1.4 var(--mono); text-transform: none; letter-spacing: 0; } @@ -3247,8 +8084,8 @@ th.num { .prose .img-layer p { margin: 0 0 6px; - font-size: 0.74rem; - font-weight: 700; + font-size: var(--fs-n3); + font-weight: var(--fw-bold); color: var(--muted); } @@ -3260,18 +8097,18 @@ th.num { .img-pill { display: inline-block; - padding: 3px 9px; - border-radius: 999px; + padding: var(--sp-1) var(--sp-2); + border-radius: var(--r-pill); border: 1px solid var(--line-strong); background: var(--bg); - font-size: 0.78rem; + font-size: var(--fs-n2); color: var(--ink-soft); line-height: 1.45; } .img-pill b { color: var(--ink); - font-weight: 650; + font-weight: var(--fw-semibold); } .img-arrow { @@ -3279,15 +8116,15 @@ th.num { flex-direction: column; align-items: center; gap: 2px; - font-size: 0.74rem; - font-weight: 650; + font-size: var(--fs-n3); + font-weight: var(--fw-semibold); color: var(--m-dup); text-align: center; } .img-arrow i { font-style: normal; - font-size: 1.5rem; + font-size: var(--fs-3); line-height: 1; } @@ -3296,7 +8133,7 @@ th.num { } .img-note { - border-radius: 10px; + border-radius: var(--r-3); } .img-host code { @@ -3326,9 +8163,9 @@ th.num { /* โ”€โ”€ A run on a cluster โ”€โ”€ */ .fig.hpc-fig { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); padding: 16px; } @@ -3340,9 +8177,9 @@ th.num { } .hpc-box { - border-radius: 12px; - padding: 11px 13px; - font-size: 0.82rem; + border-radius: var(--r-3); + padding: var(--sp-3); + font-size: var(--fs-n2); line-height: 1.5; border: 1px solid var(--line-strong); background: var(--bg); @@ -3358,10 +8195,10 @@ th.num { } .prose .hpc-title { - font-size: 0.72rem; - font-weight: 750; + font-size: var(--fs-n3); + font-weight: var(--fw-bold); text-transform: uppercase; - letter-spacing: 0.06em; + letter-spacing: var(--tr-caps); color: var(--muted); } @@ -3378,12 +8215,12 @@ th.num { } .prose .hpc-small { - font-size: 0.76rem; + font-size: var(--fs-n3); color: var(--muted); } .hpc-small code { - font-size: 0.72rem; + font-size: var(--fs-n3); } .hpc-arrow { @@ -3404,32 +8241,32 @@ th.num { .hpc-tasks span { display: block; - padding: 3px 8px; - border-radius: 7px; + padding: var(--sp-1) var(--sp-2); + border-radius: var(--r-2); background: color-mix(in srgb, var(--m-dup) 12%, transparent); - font-size: 0.76rem; + font-size: var(--fs-n3); } .hpc-shared { - margin-top: 14px; + margin-top: var(--sp-4); border-style: dashed; } .hpc-places { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); - gap: 8px 14px; + gap: var(--sp-2) var(--sp-4); } .hpc-places span { - font-size: 0.76rem; + font-size: var(--fs-n3); color: var(--ink-soft); } .hpc-places .hpc-key { display: block; width: fit-content; - font-size: 0.74rem; + font-size: var(--fs-n3); margin-bottom: 2px; } @@ -3454,10 +8291,10 @@ th.num { /* โ”€โ”€ The report, section by section โ”€โ”€ */ .fig.report-map { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; - padding: 14px 16px 12px; + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); + padding: var(--sp-4); } .prose .rp-head { @@ -3466,7 +8303,7 @@ th.num { justify-content: space-between; align-items: center; gap: 10px; - font: 600 0.8rem/1.3 var(--mono); + font: var(--fw-semibold) var(--fs-n2)/var(--lh-label) var(--mono); color: var(--muted); } @@ -3480,18 +8317,22 @@ th.num { display: flex; flex-direction: column; border: 1px solid var(--line); - border-radius: 12px; + border-radius: var(--r-3); overflow: hidden; text-decoration: none; color: var(--ink-soft); background: var(--bg); - transition: border-color 0.2s ease, transform 0.2s ease; + transition: border-color var(--dur-state) var(--ease), background-color var(--dur-state) var(--ease); } -.prose a.rp-card:hover { - border-color: var(--accent); - color: var(--ink-soft); - transform: translateY(-2px); +/* A card that is a link answers a pointer as every link card does: an + accent edge on a raised fill, no lift. */ +@media (hover: hover) { + .prose a.rp-card:hover { + border-color: var(--accent-line); + background: var(--surface-2); + color: var(--ink-soft); + } } .rp-card img { @@ -3509,28 +8350,29 @@ th.num { display: flex; flex-direction: column; gap: 3px; - padding: 10px 12px 12px; - font-size: 0.8rem; + padding: var(--sp-3); + font-size: var(--fs-n2); line-height: 1.45; } .rp-text b { color: var(--ink); - font-size: 0.9rem; + font-size: var(--fs-n1); } /* โ”€โ”€ The command builder, in a page of the docs โ”€โ”€ */ .fig.builder-fig { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); padding: 16px; } .prose .builder-fig .builder { grid-template-columns: minmax(0, 1fr); - gap: 14px; + gap: var(--sp-4); + margin-top: 0; } /* The command is the point of the builder: its button always shows. */ @@ -3539,16 +8381,23 @@ th.num { } .prose .builder-fig .explain code { - align-self: start; line-height: 1.4; } +.prose .builder-fig .cmd-head { + margin: 0 0 8px; +} + +.prose .builder-fig .cmd-head ~ .cmd-head { + margin-top: 20px; +} + /* โ”€โ”€ When a task fails โ”€โ”€ */ .fig.fail-fig { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); padding: 16px; } @@ -3559,12 +8408,12 @@ th.num { } .fail-lane { - border-radius: 12px; - padding: 12px 14px; + border-radius: var(--r-3); + padding: var(--sp-3) var(--sp-4); background: var(--bg); border: 1px solid var(--line-strong); border-top: 3px solid var(--warn); - font-size: 0.84rem; + font-size: var(--fs-n2); } .fail-lane.fail-other { @@ -3573,7 +8422,7 @@ th.num { .prose .fail-when { margin: 0 0 8px; - font-weight: 700; + font-weight: var(--fw-bold); color: var(--ink); } @@ -3605,7 +8454,7 @@ th.num { border-radius: 50%; display: grid; place-items: center; - font: 700 0.72rem/1 var(--font); + font: var(--fw-bold) var(--fs-n3)/1 var(--font); background: color-mix(in srgb, var(--warn) 22%, transparent); color: var(--ink); } @@ -3626,7 +8475,7 @@ th.num { .prose .fail-except { margin: 12px 0 0; - font-size: 0.82rem; + font-size: var(--fs-n2); color: var(--ink-soft); } @@ -3639,10 +8488,10 @@ th.num { /* โ”€โ”€ The overview: the five outcomes, drawn small โ”€โ”€ */ .fig.outcomes { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; - padding: 14px 16px 12px; + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); + padding: var(--sp-4); } .fig.outcomes .legend { @@ -3661,17 +8510,19 @@ th.num { gap: 8px; padding: 12px; border: 1px solid var(--line); - border-radius: 12px; + border-radius: var(--r-3); background: var(--bg); color: var(--ink-soft); text-decoration: none; - transition: border-color 0.2s ease, transform 0.2s ease; + transition: border-color var(--dur-state) var(--ease), background-color var(--dur-state) var(--ease); } -.prose a.oc:hover { - border-color: var(--accent); - color: var(--ink-soft); - transform: translateY(-2px); +@media (hover: hover) { + .prose a.oc:hover { + border-color: var(--accent-line); + background: var(--surface-2); + color: var(--ink-soft); + } } .oc-head { @@ -3679,7 +8530,7 @@ th.num { align-items: center; gap: 8px; color: var(--ink); - font-size: 0.92rem; + font-size: var(--fs-n1); } .oc-fig { @@ -3689,19 +8540,21 @@ th.num { } .oc-text { - font-size: 0.82rem; + font-size: var(--fs-n2); line-height: 1.45; } .oc-more { margin-top: auto; - font-size: 0.8rem; - font-weight: 650; + font-size: var(--fs-n2); + font-weight: var(--fw-semibold); color: var(--accent); } +/* The arrow is drawn, not read: its alternative text is empty. */ .oc-more::after { content: " \2192"; + content: " \2192" / ""; } .oc.oc-harder { @@ -3710,8 +8563,8 @@ th.num { gap: 8px; padding: 12px; border: 1px dashed var(--line-strong); - border-radius: 12px; - font-size: 0.86rem; + border-radius: var(--r-3); + font-size: var(--fs-n2); } .prose .oc-harder ul { @@ -3727,31 +8580,30 @@ th.num { .fig.sheet-fig, .fig.gauges { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); padding: 16px; } -.sheet { +.sheet-fig .sheet { display: grid; - gap: 14px; + grid-template-columns: minmax(0, 1fr); + gap: var(--sp-4); } .sheet-file { border: 1px solid var(--code-border); background: var(--code-bg); - border-radius: 10px; - padding: 10px 12px 12px; + border-radius: var(--r-3); + padding: var(--sp-3); min-width: 0; } .prose .sheet-name { margin: 0 0 8px; - font: 600 0.74rem/1 var(--mono); + font: var(--fw-semibold) var(--fs-n3)/1 var(--mono); color: var(--code-muted); - text-transform: uppercase; - letter-spacing: 0.06em; display: flex; align-items: center; gap: 8px; @@ -3760,7 +8612,7 @@ th.num { .prose .sheet-table { margin: 0; border: 0; - font: 0.8rem/1.5 var(--mono); + font: var(--fs-n2)/1.5 var(--mono); color: var(--code-ink); width: 100%; border-collapse: collapse; @@ -3779,9 +8631,11 @@ th.num { vertical-align: top; } +/* A column's name keeps its numbered badge beside it. */ .prose .sheet-table th { color: var(--code-flag); - font-weight: 700; + font-weight: var(--fw-bold); + white-space: nowrap; } .prose .sheet-table .sheet-tab { @@ -3789,13 +8643,17 @@ th.num { background: color-mix(in srgb, var(--code-value) 14%, transparent); text-align: center; width: 2.2em; - border-radius: 4px; + border-radius: var(--r-1); } .sheet-path { overflow-wrap: anywhere; } +.sheet-path .nw { + white-space: nowrap; +} + .sheet-n { display: inline-grid; place-items: center; @@ -3805,7 +8663,7 @@ th.num { border-radius: 50%; background: var(--accent); color: var(--on-accent); - font: 700 0.68rem/1 var(--font); + font: var(--fw-bold) var(--fs-n3)/1 var(--font); vertical-align: 1px; } @@ -3819,7 +8677,7 @@ th.num { padding: 0; list-style: none; counter-reset: note; - font-size: 0.86rem; + font-size: var(--fs-n2); display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 12px; @@ -3844,7 +8702,7 @@ th.num { place-items: center; background: var(--accent); color: var(--on-accent); - font: 700 0.68rem/1 var(--font); + font: var(--fw-bold) var(--fs-n3)/1 var(--font); } .prose .sheet-notes li:nth-child(2)::before { @@ -3855,12 +8713,12 @@ th.num { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); gap: 12px; - margin-top: 14px; + margin-top: var(--sp-4); } .prose .sheet-when { margin: 0 0 6px; - font-size: 0.82rem; + font-size: var(--fs-n2); color: var(--muted); } @@ -3868,15 +8726,17 @@ th.num { white-space: nowrap; } +/* A little less room on the right keeps main.nf's longest line whole in + the two columns of a wide screen. */ .prose .sheet-check pre { margin: 0; - padding: 10px 12px; - border-radius: 9px; + padding: var(--sp-3) var(--sp-2) var(--sp-3) var(--sp-3); + border-radius: var(--r-3); background: var(--code-bg); border: 1px solid var(--code-border); border-left: 3px solid var(--danger); color: var(--code-ink); - font: 0.74rem/1.55 var(--mono); + font: var(--fs-n3)/var(--lh-read) var(--mono); white-space: pre-wrap; overflow-wrap: anywhere; } @@ -3887,7 +8747,7 @@ th.num { .prose .gauge-name { margin: 0 0 30px; - font-size: 0.88rem; + font-size: var(--fs-n2); display: flex; gap: 8px; align-items: baseline; @@ -3895,13 +8755,13 @@ th.num { .gauge-name span { color: var(--muted); - font-size: 0.78rem; + font-size: var(--fs-n2); } .gauge-track { position: relative; height: 12px; - border-radius: 6px; + border-radius: var(--r-2); background: var(--bg-subtle); } @@ -3912,11 +8772,11 @@ th.num { } .gauge-zone:first-child { - border-radius: 6px 0 0 6px; + border-radius: var(--r-2) 0 0 var(--r-2); } .gauge-zone:nth-last-child(1 of .gauge-zone) { - border-radius: 0 6px 6px 0; + border-radius: 0 var(--r-2) var(--r-2) 0; } .gz-ok { @@ -3948,7 +8808,8 @@ th.num { width: 0; } -.gauge-mark i { +/* The mark's dot; an i inside its label is a species' italics. */ +.gauge-mark > i { position: absolute; left: -6px; top: -6px; @@ -3965,11 +8826,16 @@ th.num { transform: translateX(-50%); white-space: nowrap; font-style: normal; - font-size: 0.74rem; - font-weight: 600; + font-size: var(--fs-n3); + font-weight: var(--fw-semibold); color: var(--ink-soft); } +/* The label is set upright; a species in it keeps its italics. */ +.gauge-mark em i { + font-style: italic; +} + .gm-up em { bottom: 12px; } @@ -3990,7 +8856,7 @@ th.num { position: relative; height: 16px; margin-top: 4px; - font: 0.7rem/1 var(--mono); + font: var(--fs-n3)/1 var(--mono); color: var(--muted); } @@ -4005,14 +8871,25 @@ th.num { white-space: nowrap; } -.gauge-ticks span:first-child { +.gauge-ticks .t-start { transform: none; } -.gauge-ticks span:last-child { +.gauge-ticks .t-end { transform: translateX(-100%); } +/* A gauge narrower than its ticks need shows the major ones only. */ +.gauge { + container-type: inline-size; +} + +@container (max-width: 640px) { + .gauge-ticks .minor { + display: none; + } +} + .prose .gauge-list { display: none; } @@ -4025,8 +8902,7 @@ th.num { } @media (max-width: 600px) { - .gauge-mark em, - .gauge-ticks .minor { + .gauge-mark em { display: none; } @@ -4043,7 +8919,7 @@ th.num { margin: 8px 0 0; padding: 0; list-style: none; - font-size: 0.78rem; + font-size: var(--fs-n2); color: var(--ink-soft); } @@ -4067,33 +8943,50 @@ th.num { /* โ”€โ”€ The benchmark, on the front page โ”€โ”€ */ +/* A table for everyone: its rows and cells carry their ARIA roles, and the + bars only draw what the numbers in them say. Each bar is AMAP; the part + of it Bakta finds on its own, scored by the same rules, is shaded up to a + line, and its number is printed under the bar. Set as a table in a paper + is: a heavy rule over it and under it, a hairline under its head and + between its rows, no boxes. */ .proof { display: grid; - gap: 8px; - max-width: 980px; + max-width: none; + border-top: 2px solid var(--ink-3); + border-bottom: 2px solid var(--ink-3); + font-variant-numeric: tabular-nums; } .pf-row { display: grid; - grid-template-columns: minmax(0, 1.1fr) minmax(0, 1fr) minmax(0, 1fr); - gap: 16px; - align-items: center; - padding: 10px 14px; - border: 1px solid var(--line); - border-radius: 12px; - background: var(--bg-elev); + grid-template-columns: minmax(0, 1fr) minmax(0, 1.25fr) minmax(0, 1.25fr); + gap: var(--sp-6); + align-items: start; + padding: var(--sp-4) 0; + border-top: 1px solid var(--line-1); +} + +/* On a wide screen the genomes' names take the margin column, so the bars + start where the chapter's text does. */ +@media (min-width: 941px) { + .pf-row { + grid-template-columns: var(--col-margin) minmax(0, 1fr) minmax(0, 1fr); + column-gap: var(--col-gap); + } } .pf-headrow { - border: 0; - background: none; - padding-top: 0; - padding-bottom: 0; - font-size: 0.78rem; - font-weight: 700; - color: var(--muted); - text-transform: uppercase; - letter-spacing: 0.05em; + padding: var(--sp-3) 0; + border-top: 0; + border-bottom: 1px solid var(--line-2); + font-size: var(--fs-n2); + font-weight: var(--fw-semibold); + line-height: var(--lh-ui); + color: var(--ink-3); +} + +.pf-headrow + .pf-row { + border-top: 0; } .pf-name { @@ -4103,61 +8996,100 @@ th.num { } .pf-name b { - font-size: 0.95rem; + font-size: var(--fs-0); + font-weight: var(--fw-semibold); + color: var(--ink-1); } .pf-name small { - color: var(--muted); - font-size: 0.78rem; + color: var(--ink-3); + font-size: var(--fs-n2); } .pf-bar { + display: grid; + gap: var(--sp-1); + min-width: 0; +} + +.pf-track { position: relative; display: block; - height: 26px; - border-radius: 7px; - background: var(--bg-subtle); + height: 28px; + border-radius: var(--r-2); + background: var(--surface-3); overflow: hidden; } -.pf-bar i { +.pf-track > i { position: absolute; inset: 0 auto 0 0; +} + +.pf-fill { + background: color-mix(in srgb, var(--amap) 45%, transparent); +} + +/* The light theme keeps the stronger fill: its dark ink reads on it. */ +:root[data-theme="light"] .pf-exact .pf-fill { background: color-mix(in srgb, var(--amap) 55%, transparent); } -.pf-changed i { +.pf-changed .pf-fill { background: color-mix(in srgb, var(--ref) 50%, transparent); } -.pf-bar b, -.pf-bar span { +/* Bakta alone: a shade over the part of the bar it reaches, ended by a line, + so the stretch only AMAP finds stays bright. */ +.pf-ghost { + background: color-mix(in srgb, var(--bg) 52%, transparent); + box-shadow: inset -2px 0 0 var(--ink-soft); +} + +:root[data-theme="light"] .pf-ghost { + background: color-mix(in srgb, var(--bg-elev) 58%, transparent); +} + +.pf-track b { position: relative; display: block; - padding: 0 10px; - line-height: 26px; - font: 650 0.8rem/26px var(--mono); + padding: 0 var(--sp-3); + font: var(--fw-semibold) var(--fs-n2)/28px var(--mono); color: var(--ink); } -.pf-none { - background: none; - border: 1px dashed var(--line); +.pf-bar small { + padding-left: 2px; + font-size: var(--fs-n2); + color: var(--ink-3); } -.pf-none span { - font-family: var(--font); - font-weight: 500; - color: var(--muted); +.pf-none .pf-track { + background: none; + border: 1px dashed var(--line-2); + padding: 0 var(--sp-3); + line-height: 26px; + font-size: var(--fs-n2); + color: var(--ink-3); } @media (max-width: 760px) { .pf-row { grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); + gap: var(--sp-2) var(--sp-3); + padding: var(--sp-3) 0; } .pf-name { grid-column: 1 / -1; + flex-direction: row; + flex-wrap: wrap; + gap: 0 8px; + align-items: baseline; + } + + .pf-name b { + font-size: var(--fs-n1); } .pf-headrow span:first-child { @@ -4165,29 +9097,136 @@ th.num { } } +/* On a phone the count sits over a slim bar, so the line where Bakta stops + never runs through its digits. */ +@media (max-width: 480px) { + .pf-bar:not(.pf-none) .pf-track { + height: auto; + padding-bottom: 12px; + background: none; + overflow: visible; + } + + .pf-bar:not(.pf-none) .pf-track::after { + content: ""; + position: absolute; + left: 0; + right: 0; + bottom: 0; + height: 8px; + border-radius: var(--r-1); + background: var(--surface-3); + } + + .pf-bar:not(.pf-none) .pf-track > i { + inset: auto auto 0 0; + height: 8px; + z-index: 1; + } + + .pf-bar:not(.pf-none) .pf-fill { + border-radius: var(--r-1); + } + + .pf-bar:not(.pf-none) .pf-track b { + padding: 0 2px; + line-height: 1.6; + } +} + /* โ”€โ”€ The page that is not there โ”€โ”€ */ +/* A plate, as the drawings of the other pages stand on. */ .lost { max-width: 560px; - margin: 26px 0 30px; - padding: 16px 18px 10px; - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; + margin: var(--sp-6) 0 var(--sp-7); + padding: var(--sp-4) var(--sp-5) var(--sp-3); + background: var(--surface-2); + border: 1px solid var(--line-2); + border-radius: var(--r-5); + box-shadow: var(--shadow-2); } .lost figcaption { margin-top: 6px; - font-size: 0.84rem; + font-size: var(--fs-n2); color: var(--muted); } +/* The drawing keeps a size at which its words read (at least 9 px): wide, + its rows' names beside them; on a phone, a narrower one with each name + over its row, which fits the screen whole. */ +.lost-scroll { + overflow-x: auto; +} + +.lost-scroll svg { + display: block; + width: 100%; + min-width: 520px; +} + +.lost-scroll svg.lost-narrow { + display: none; +} + +@media (max-width: 600px) { + .lost-scroll svg.lost-wide { + display: none; + } + + .lost-scroll svg.lost-narrow { + display: block; + min-width: 0; + } +} + +.lost-hero { + padding-bottom: 72px; +} + +.lost-hero .nearest { + margin-top: var(--sp-5); + font-size: var(--fs-0); + color: var(--ink-soft); +} + +.lost-hero .nearest a { + font-weight: var(--fw-semibold); +} + +.lost-search { + max-width: 420px; + margin-top: var(--sp-5); +} + +/* "Search for it:" names the box under it, and keeps to it. */ +.lost-hero .search-label + .lost-search { + margin-top: var(--sp-2); +} + +/* Its results open within its own width, over the drawing, not past it. */ +.lost-search .docs-results { + right: 0; +} + +/* On a tall screen the footer still closes the page. */ +body.lost-page { + display: flex; + flex-direction: column; + min-height: 100vh; +} + +body.lost-page > main { + flex: 1 0 auto; +} + /* โ”€โ”€ Duplications: a family and a pair of communities, as networks โ”€โ”€ */ .fig.dup-fig { - background: var(--bg-elev); - border: 1px solid var(--line); - border-radius: 14px; + background: var(--surface-1); + border: 1px solid var(--line-2); + border-radius: var(--r-4); padding: 16px 16px 12px; } @@ -4195,6 +9234,39 @@ th.num { position: relative; } +/* The room each network takes, kept before the script draws it: the same + proportions as netSvg in figures.js. The figure itself is not a container, + because a container scopes the figure counter its caption steps. */ +.dup-side { + container-type: inline-size; +} + +.dup-fig > .case-fig:empty { + aspect-ratio: 100 / 105; +} + +@media (min-width: 600px) { + .dup-fig > .case-fig:empty { + aspect-ratio: 100 / 62; + max-height: 420px; + } +} + +.dup-side > .case-fig:empty { + height: 92cqw; +} + +@container (min-width: 520px) { + .dup-side > .case-fig:empty { + height: min(380px, 56cqw); + } +} + +.dup-head:empty, +.dup-bridge:empty { + min-height: calc(1.55em + 14px); +} + .net { display: block; width: 100%; @@ -4222,11 +9294,20 @@ th.num { stroke-width: 14px; } +.net-pill-hit { + fill: transparent; +} + .net-link { - cursor: default; outline: none; } +/* Each protein and link has its numbers to show: a target, as a gene is. */ +.net-link, +.net-node { + cursor: pointer; +} + .net-link.kept .net-line { stroke: var(--m-dup); } @@ -4254,7 +9335,7 @@ th.num { } .net-pill { - fill: var(--bg-elev); + fill: var(--halo); stroke-width: 1px; } @@ -4268,7 +9349,7 @@ th.num { .net-pct { font-size: 10.5px; - font-weight: 650; + font-weight: var(--fw-semibold); fill: var(--ink-soft); font-variant-numeric: tabular-nums; } @@ -4283,7 +9364,7 @@ th.num { } .net-node.out circle { - fill: var(--bg-elev); + fill: var(--halo); stroke: var(--drop); stroke-dasharray: 3 3; } @@ -4297,19 +9378,41 @@ th.num { outline: none; } +/* The ring a protein shows when the keyboard is on it, and only then. */ +.net-node circle.net-ring { + fill: none; + stroke: transparent; + stroke-dasharray: none; +} + +.net-node:focus-visible circle.net-ring { + stroke: var(--ink); + stroke-width: 2px; +} + +.net-link:focus-visible .net-hit { + stroke: color-mix(in srgb, var(--ink) 22%, transparent); +} + .net-name { font-size: 11.5px; - font-weight: 650; + font-weight: var(--fw-semibold); fill: var(--ink); paint-order: stroke; - stroke: var(--bg-elev); + stroke: var(--halo); stroke-width: 3px; stroke-linejoin: round; } .net-node.out .net-name { fill: var(--muted); - font-weight: 600; + font-weight: var(--fw-semibold); +} + +/* Inside the family's outline a name's ring takes the outline's tint, so + no darker patch shows round its letters. */ +.net-node.in .net-name { + stroke: color-mix(in srgb, var(--m-dup) 9%, var(--halo)); } .chip.link-kept, @@ -4346,12 +9449,12 @@ th.num { .dup-sides { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); - gap: 6px 18px; + gap: var(--sp-2) var(--sp-4); } .dup-head { margin: 0 0 4px !important; - font-size: 0.86rem; + font-size: var(--fs-n2); color: var(--muted); display: flex; flex-wrap: wrap; @@ -4361,17 +9464,17 @@ th.num { .dup-head b { color: var(--ink); - font-weight: 650; + font-weight: var(--fw-semibold); } .dup-bridge { grid-column: 1 / -1; position: relative; text-align: center; - font-size: 0.82rem; - font-weight: 650; + font-size: var(--fs-n2); + font-weight: var(--fw-semibold); color: var(--m-dup); - padding-top: 14px; + padding-top: var(--sp-4); } .dup-bridge::before { @@ -4383,7 +9486,7 @@ th.num { height: 10px; border: 2px dashed color-mix(in srgb, var(--m-dup) 70%, transparent); border-top: 0; - border-radius: 0 0 10px 10px; + border-radius: 0 0 var(--r-3) var(--r-3); } .dup-metrics { @@ -4400,6 +9503,18 @@ th.num { font-variant-numeric: tabular-nums; } +/* On a phone the cells take less room, so both genomes' columns show. */ +@media (max-width: 600px) { + .prose .dup-metrics td, + .prose .dup-metrics th { + padding: var(--sp-1) var(--sp-2); + } + + .prose .dup-metrics :is(td, th):first-child { + padding-left: 0; + } +} + .dup-metrics code { white-space: nowrap; } @@ -4412,8 +9527,8 @@ th.num { .dup-metrics .th-sub { display: block; - font-weight: 500; - font-size: 0.74rem; + font-weight: var(--fw-medium); + font-size: var(--fs-n3); color: var(--muted); } @@ -4440,6 +9555,104 @@ th.num { } } +/* โ”€โ”€ Touch, again โ”€โ”€ + A finger needs a 40px target. Set here, after every control's own rules, + so a later rule of the same weight cannot take it back. */ +@media (pointer: coarse) { + .copy, + .mini-btn, + .tab, + .seg button, + .case .seg button, + .how-group .seg a, + .metro-opts label, + .icon-btn, + .gh-link, + .search-btn, + .sheet-close { + min-height: 40px; + min-width: 40px; + } + + .sheet-close { + top: 4px; + } + + /* A block's head holds the taller Copy with 4px above and below it, and + so does the strip over a block with no head. */ + .code > .code-title { + min-height: 48px; + } + + .code-title ~ .copy { + top: 4px; + } + + .nav a { + padding-block: 10px; + } + + .docs-nav li a, + .toc-inline a { + padding-block: 9px; + } + + .foot ul { + gap: 0; + } + + .foot li a { + display: inline-block; + padding-block: 11px; + } + + /* The hero's phases: a taller press, the same thin bar. */ + .hero-seg { + height: 40px; + } + + .hero-seg::before, + .hero-seg i { + top: 18px; + } + + /* Links set in a line of text grow round it without moving it. */ + .to-top a, + .numbers-head a, + .crumb a { + display: inline-block; + } + + .to-top a, + .edit a, + .numbers-head a, + .crumb a { + padding-block: 11px; + margin-block: -11px; + } + + .prose .anchor { + padding: 6px 10px; + margin: -6px -10px; + } +} + +@media (pointer: coarse) and (max-width: 860px) { + .prose .hx > .anchor { + margin-left: calc(0.3em - 10px); + } +} + +@media (pointer: coarse) and (max-width: 600px) { + .code:not(.out):not(:has(> .code-title)) { + padding-top: 48px; + } + + .code:not(.out):not(:has(> .code-title)) > .copy { + top: 4px; + } +} + @media (prefers-reduced-motion: reduce) { html { scroll-behavior: auto; @@ -4454,3 +9667,72 @@ th.num { transition: none; } } + +@media (prefers-reduced-motion: reduce) { + .trk .step-off, + .trk [data-from], + .trk .rib-g, + .trk .row-frame, + .trk .hot, + .how-progress i, + .metro-lines span, + .metro .exit, + .tally b, + .tally > div { + transition: none; + } + + .demo.running .hero-seg.now i { + animation: none; + transform: scaleX(1); + } +} + +/* In forced colours (a high-contrast theme) the drawings keep their own + palette, which is what they mean and already reads in either theme, on + the card colour it was made for. */ +@media (forced-colors: active) { + .demo, + .fig, + .case-fig, + .how-stage-card, + .metro-wrap, + .net, + .legend .chip { + forced-color-adjust: none; + } + + .case-fig, + .net { + background: var(--bg-elev); + } + + /* A chosen, pressed or current control is drawn by its background and + shadow, which forced colours take away: in their place, the system's + highlight. */ + .seg button:is([aria-pressed="true"], [aria-checked="true"]), + .how-group .seg a[aria-current="true"], + .tab[aria-selected="true"], + .how-steps button[aria-current="step"], + .tally > [aria-pressed="true"], + .docs-results a.active, + .docs-nav .page-toc a.active[aria-current], + .rail a.active[aria-current], + .toc-inline a.active, + .sheet-toc a.active, + .docs-nav li a[aria-current="page"], + .nav a[aria-current] { + forced-color-adjust: none; + background: Highlight; + color: HighlightText; + border-color: Highlight; + } + + .seg button:is([aria-pressed="true"], [aria-checked="true"]) *, + .how-group .seg a[aria-current="true"] *, + .how-steps button[aria-current="step"] *, + .docs-results a.active *, + .rail a.active * { + color: HighlightText; + } +} diff --git a/docs/assets/site.js b/docs/assets/site.js index 5d73801..6e87a04 100644 --- a/docs/assets/site.js +++ b/docs/assets/site.js @@ -1,13 +1,48 @@ /* AMAP website: theme, tabs, copy buttons, the command builder, the embedded - report and the documentation pages' search and contents. Plain JavaScript, - no build step. The drawings are in figures.js. */ + report, the search on every page, and the documentation pages' contents. + Plain JavaScript, no build step. The drawings are in figures.js. */ (function () { "use strict"; + // The site's root, read from this script's own address: the search index and + // the pages it lists sit there, whatever folder this page is in (the 404 page + // is served from any path). + var SITE_SCRIPT = document.currentScript || document.querySelector('script[src*="assets/site.js"]'); + var ROOT = SITE_SCRIPT ? SITE_SCRIPT.src.replace(/assets\/site\.js(?:[?#].*)?$/, "") : ""; + + function reducedMotion() { + return !!(window.matchMedia && window.matchMedia("(prefers-reduced-motion: reduce)").matches); + } + + // Storage can be missing or refused (private windows, blocked cookies); the + // page works without it. + function sessionGet(key) { + try { + return JSON.parse(sessionStorage.getItem(key)); + } catch (e) { + return null; + } + } + + function sessionSet(key, value) { + try { + if (value === null) sessionStorage.removeItem(key); + else sessionStorage.setItem(key, JSON.stringify(value)); + } catch (e) { + /* nothing to remember with */ + } + } + /* โ”€โ”€ Theme โ”€โ”€ */ - // Dark unless the reader chose light. + // Dark unless the reader chose light. The button is a toggle for the light + // theme: pressed when light is on, and its icon shows what a press gives you. + // The browser's own bar (theme-color) follows the page. var THEME_KEY = "amap.theme"; + var THEME_COLOR = { dark: "#0a0e0d", light: "#f5f7f6" }; + var MOON = + ''; function currentTheme() { return document.documentElement.dataset.theme === "light" ? "light" : "dark"; @@ -21,16 +56,83 @@ } } - document.querySelectorAll("[data-theme-toggle]").forEach(function (button) { + var themeToggles = document.querySelectorAll("[data-theme-toggle]"); + + function syncTheme() { + var light = currentTheme() === "light"; + themeToggles.forEach(function (button) { + button.setAttribute("aria-pressed", light ? "true" : "false"); + button.title = light ? "Switch to the dark theme" : "Switch to the light theme"; + }); + var meta = document.querySelector('meta[name="theme-color"]'); + if (meta) meta.setAttribute("content", THEME_COLOR[light ? "light" : "dark"]); + } + + themeToggles.forEach(function (button) { + var sun = button.querySelector("svg"); + if (sun) sun.classList.add("i-sun"); + button.insertAdjacentHTML("beforeend", MOON); + button.setAttribute("aria-label", "Light theme"); button.addEventListener("click", function () { var next = currentTheme() === "dark" ? "light" : "dark"; - document.documentElement.dataset.theme = next; + // Every surface takes the new theme at once: the controls' own + // transitions are held for the swap, so none fades in behind the page. + var apply = function () { + var html = document.documentElement; + html.classList.add("theme-swap"); + html.dataset.theme = next; + syncTheme(); + void document.body.offsetWidth; + requestAnimationFrame(function () { + requestAnimationFrame(function () { + html.classList.remove("theme-swap"); + }); + }); + }; + // A short cross-fade where the browser can draw one, none under reduced motion. + if (document.startViewTransition && !reducedMotion()) document.startViewTransition(apply); + else apply(); remember(next); }); }); + syncTheme(); + // Anything else that sets the theme (another tab, a test) keeps the button + // and the browser bar in step. + if (window.MutationObserver) { + new MutationObserver(syncTheme).observe(document.documentElement, { attributes: true, attributeFilter: ["data-theme"] }); + } + + /* โ”€โ”€ Status โ”€โ”€ */ + + // One line that screen readers read out, for changes that happen away from + // focus: a copy, the command builder's new command, the number of results. + var statusLine = document.createElement("div"); + statusLine.className = "visually-hidden"; + statusLine.setAttribute("role", "status"); + document.body.insertBefore(statusLine, document.body.firstChild); + var statusTimer = 0; + + function announce(text, delay) { + clearTimeout(statusTimer); + statusLine.textContent = ""; + // Emptied first, so the same words twice are still read twice. + statusTimer = setTimeout(function () { + statusLine.textContent = text; + }, delay || 60); + } /* โ”€โ”€ Tabs โ”€โ”€ */ + // A file excerpt too wide for a phone scrolls; it opens on what it is + // there to show, its first highlighted mark, with a little before it. + function revealMark(panel) { + var pre = panel && panel.querySelector("pre"); + var mark = pre && pre.querySelector("mark"); + if (!mark || pre.scrollWidth <= pre.clientWidth + 1) return; + var left = mark.getBoundingClientRect().left - pre.getBoundingClientRect().left + pre.scrollLeft; + pre.scrollLeft = Math.max(0, left - pre.clientWidth * 0.3); + } + document.querySelectorAll('[role="tablist"]').forEach(function (list) { var tabs = Array.prototype.slice.call(list.querySelectorAll('[role="tab"]')); function select(tab) { @@ -40,24 +142,101 @@ t.tabIndex = on ? 0 : -1; var panel = document.getElementById(t.getAttribute("aria-controls")); if (panel) panel.hidden = !on; + if (on) revealMark(panel); }); } + tabs.forEach(function (tab) { + if (tab.getAttribute("aria-selected") === "true") revealMark(document.getElementById(tab.getAttribute("aria-controls"))); + }); tabs.forEach(function (tab, i) { tab.tabIndex = tab.getAttribute("aria-selected") === "true" ? 0 : -1; tab.addEventListener("click", function () { select(tab); }); + // The arrows move along the row, Home and End to its ends, as the + // builder's radio groups do. tab.addEventListener("keydown", function (event) { - var step = event.key === "ArrowRight" ? 1 : event.key === "ArrowLeft" ? -1 : 0; - if (!step) return; + var to = { ArrowRight: i + 1, ArrowLeft: i - 1, Home: 0, End: tabs.length - 1 }[event.key]; + if (to === undefined) return; event.preventDefault(); - var next = tabs[(i + step + tabs.length) % tabs.length]; + var next = tabs[(to + tabs.length) % tabs.length]; select(next); next.focus(); }); }); }); + /* โ”€โ”€ Sideways scrollers โ”€โ”€ */ + + // On a phone the case cards, the story chips and the metro map scroll + // sideways. What the keyboard focuses there comes into view: a case's + // whole card, anything else just enough. Not on a screen wide enough for + // them, where nothing scrolls. + ["cases", "how-group", "metro-scroll"].forEach(function (name) { + document.querySelectorAll("." + name).forEach(function (box) { + box.addEventListener("focusin", function (event) { + var scroller = name === "how-group" ? event.target.closest(".seg") : box; + if (!scroller || scroller.scrollWidth <= scroller.clientWidth + 1) return; + var card = name === "cases" && event.target.closest(".case"); + (card || event.target).scrollIntoView({ block: "nearest", inline: card ? "start" : "nearest" }); + }); + }); + }); + + /* โ”€โ”€ Scroll regions โ”€โ”€ */ + + // A table, a command or a program's output too wide for the screen + // scrolls sideways. While it does, it is a tab stop with a name, so the + // keyboard can scroll it; while it fits, it is neither. + function scrollRegion(el) { + var scrolls = el.scrollWidth > el.clientWidth + 1; + var was = el.hasAttribute("data-scroll-region"); + if (scrolls === was) return; + if (scrolls) { + el.setAttribute("data-scroll-region", ""); + el.setAttribute("tabindex", "0"); + el.setAttribute("role", "region"); + if (!el.hasAttribute("aria-labelledby") && !el.hasAttribute("aria-label")) { + var code = el.closest(".code"); + var title = code && code.querySelector(".code-title"); + var kind = el.classList.contains("table-wrap") + ? "Table" + : title + ? title.textContent.trim() + : el.closest(".cmd, .builder") || el.querySelector(".ln") + ? "Command" + : el.closest(".excerpt") + ? "File excerpt" + : el.classList.contains("out") || el.closest(".out") + ? "Output" + : "Text"; + el.setAttribute("aria-label", kind + ", scrolls sideways"); + el.setAttribute("data-scroll-label", ""); + } + } else { + el.removeAttribute("data-scroll-region"); + el.removeAttribute("tabindex"); + el.removeAttribute("role"); + if (el.hasAttribute("data-scroll-label")) { + el.removeAttribute("aria-label"); + el.removeAttribute("data-scroll-label"); + } + } + } + var scrollers = document.querySelectorAll(".table-wrap, .code pre, .out, .excerpt pre"); + if ("ResizeObserver" in window) { + var watchScroll = new ResizeObserver(function (records) { + records.forEach(function (r) { + scrollRegion(r.target); + }); + }); + scrollers.forEach(function (el) { + watchScroll.observe(el); + }); + } else { + scrollers.forEach(scrollRegion); + } + /* โ”€โ”€ Copy buttons โ”€โ”€ */ function plainText(node) { @@ -68,40 +247,75 @@ return clone.textContent.replace(/\s+$/, "") + "\n"; } + // The clipboard API when the page may use it, the old way when it may not + // (a page opened from a clone, a browser that refuses without a gesture). function copyText(text) { + var fallback = function () { + return new Promise(function (resolve, reject) { + var area = document.createElement("textarea"); + area.value = text; + area.setAttribute("readonly", ""); + area.style.position = "fixed"; + area.style.opacity = "0"; + document.body.appendChild(area); + area.select(); + try { + document.execCommand("copy") ? resolve() : reject(); + } catch (e) { + reject(e); + } + area.remove(); + }); + }; if (navigator.clipboard && window.isSecureContext) { - return navigator.clipboard.writeText(text); - } - return new Promise(function (resolve, reject) { - var area = document.createElement("textarea"); - area.value = text; - area.setAttribute("readonly", ""); - area.style.position = "fixed"; - area.style.opacity = "0"; - document.body.appendChild(area); - area.select(); - try { - document.execCommand("copy") ? resolve() : reject(); - } catch (e) { - reject(e); - } - area.remove(); - }); + return navigator.clipboard.writeText(text).catch(fallback); + } + return fallback(); } + // The button's two words share one cell, so it keeps its width when it + // says "Copied"; each click starts the 1.6 s again. Where the browser + // refuses, the block's text is selected for the reader's own Ctrl+C, and + // the button says so for as long. function addCopy(button, source) { + var label = button.textContent; + button.textContent = ""; + var ready = document.createElement("span"); + ready.className = "say-1"; + ready.textContent = label; + var said = document.createElement("span"); + said.className = "say-2"; + said.textContent = "Copied"; + button.appendChild(ready); + button.appendChild(said); + function settle() { + clearTimeout(button._t); + button._t = setTimeout(function () { + button.classList.remove("done", "ask"); + said.textContent = "Copied"; + }, 1600); + } button.addEventListener("click", function () { copyText(plainText(source)).then( function () { - button.textContent = "Copied"; + said.textContent = "Copied"; + button.classList.remove("ask"); button.classList.add("done"); - setTimeout(function () { - button.textContent = "Copy"; - button.classList.remove("done"); - }, 1600); + announce("Copied to the clipboard"); + settle(); }, function () { - button.textContent = "Select and copy"; + var selection = window.getSelection && window.getSelection(); + if (selection) selection.selectAllChildren(source); + // The keys to press where there is a keyboard; on a touch screen, + // only that the text is selected. + var touch = window.matchMedia && window.matchMedia("(hover: none)").matches; + var mac = /Mac|iP(hone|ad|od)/.test(navigator.platform || ""); + said.textContent = touch ? "Selected" : mac ? "Press Cmd+C" : "Press Ctrl+C"; + button.classList.remove("done"); + button.classList.add("ask"); + announce(touch ? "The text is selected, ready to copy" : "Selected: press " + (mac ? "Command" : "Control") + " and C to copy"); + settle(); } ); }); @@ -112,6 +326,68 @@ if (source) addCopy(button, source); }); + // A short note beside what was copied (after a heading's last word, or + // above a flag in a table). It sits over the page, in the page's own + // coordinates, so nothing moves and it scrolls with the text. + function noteBeside(target, text, above) { + var range = document.createRange(); + range.selectNodeContents(target); + var rects = range.getClientRects(); + var box = rects.length && !above ? rects[rects.length - 1] : target.getBoundingClientRect(); + var note = document.createElement("span"); + note.className = "copied-note" + (above ? " above" : ""); + note.setAttribute("aria-hidden", "true"); + note.textContent = text; + note.style.left = Math.round((above ? box.left : box.right + 10) + window.scrollX) + "px"; + note.style.top = Math.round(box.top + window.scrollY + (above ? -6 : box.height / 2)) + "px"; + document.body.appendChild(note); + // A note that would run past the window's edge (a long heading on a + // phone) sits above the line instead, inside the edge, so the page never + // widens under it. + var room = document.documentElement.clientWidth - 8; + if (note.getBoundingClientRect().right > room) { + note.classList.add("above"); + note.style.left = Math.round(Math.max(8, room - note.offsetWidth) + window.scrollX) + "px"; + note.style.top = Math.round(box.top + window.scrollY - 6) + "px"; + } + setTimeout(function () { + note.remove(); + }, 1500); + } + + // A heading's "#" copies the link to that heading, and still moves the + // address bar there. An option's flag in the Parameters tables copies the + // flag. + document.querySelectorAll('tr[id^="opt-"] > :first-child code').forEach(function (code) { + code.classList.add("copyable"); + code.title = "Click to copy"; + }); + + document.addEventListener("click", function (event) { + if (event.button || event.metaKey || event.ctrlKey || event.shiftKey || event.altKey) return; + var target = event.target.closest ? event.target : event.target.parentElement; + if (!target) return; + var anchor = target.closest("a.anchor"); + if (anchor) { + var hash = anchor.getAttribute("href") || ""; + if (hash.charAt(0) !== "#") return; + var heading = anchor.closest("h1, h2, h3, h4, h5, h6") || document.getElementById(decodeURIComponent(hash.slice(1))); + copyText(location.href.split("#")[0] + hash).then(function () { + noteBeside(heading || anchor, "Link copied"); + announce("Link copied"); + }, function () {}); + return; + } + var flag = target.closest(".copyable"); + if (flag && !String(window.getSelection ? window.getSelection() : "")) { + var text = flag.textContent.trim(); + copyText(text).then(function () { + noteBeside(flag, "Copied", true); + announce("Copied " + text); + }, function () {}); + } + }); + // Every other code block gets a button too. A one-line block wraps rather // than run under the button; a block of lines keeps its layout. function decorateCode(root) { @@ -131,11 +407,34 @@ /* โ”€โ”€ Command builder โ”€โ”€ */ + // Step 3 of the front page, and the quickstart's copy of it: two commands + // from one set of choices. The check is the run with --validate_only, so it + // looks at the databases and the reference the run will use. Every option + // the out.push lines below write is checked against main.nf and + // nextflow.config by tools/check_site.py. + if (document.querySelector("[data-builder]")) { + // The boot script in the page's head sets this before the first paint; + // a page without it still gets its choices shown. + document.documentElement.classList.add("js"); + } + + // The site's root, from this script's own address: the notes link to the + // docs from the front page and from a page a folder down. + var builderRoot = (function () { + var me = document.querySelector('script[src$="assets/site.js"]'); + return me ? me.getAttribute("src").replace("assets/site.js", "") : ""; + })(); + document.querySelectorAll("[data-builder]").forEach(function (builder) { + var checkEl = builder.querySelector("[data-check]"); var commandEl = builder.querySelector("[data-command]"); var explainEl = builder.querySelector("[data-explain]"); var noteEl = builder.querySelector("[data-builder-note]"); var choice = { engine: "docker", organism: "mtbc" }; + // Whether "Only the corrected genes" was ticked by the laptop option, not + // by the reader: unticking the laptop then unticks it too. + var onlyByLaptop = false; + var said = null; var PROFILES = { docker: { profile: "docker", says: "Every tool runs inside the AMAP image, through Docker.", db: "~/databases/eggnog" }, @@ -143,9 +442,13 @@ slurm: { profile: "singularity,slurm", says: "Each task is a SLURM job, run in the image.", db: "/shared/databases/eggnog" } }; + function box(name) { + return builder.querySelector('[data-opt="' + name + '"]'); + } + function opt(name) { - var box = builder.querySelector('[data-opt="' + name + '"]'); - return box && box.checked && !box.disabled; + var b = box(name); + return b && b.checked && !b.disabled; } function esc(text) { @@ -163,231 +466,1563 @@ out.push([["--genus", "Escherichia", "--species", '"Escherichia coli"'], "Passed to Bakta, which uses them to name genes."]); out.push([["--ref_annotation", "ref.gbk", "--ref_genome", "ref.fna"], "Your trusted reference: its annotation and the genome it describes, the same assembly."]); } - if (opt("small")) out.push([["--max_cpus", "4", "--max_memory", "12.GB"], "Ceilings: every task is clamped to them."]); - if (opt("only")) out.push([["--skip_functional_annotation"], "Stops after the correction, with the corrected genes as a table and sequences: no eggNOG, GFF3 or GenBank, duplications, pangenome or report."]); + if (opt("small")) out.push([["--max_cpus", "4", "--max_memory", "8.GB"], "Ceilings: every task is clamped to them."]); + if (opt("only")) out.push([["--skip_functional_annotation"], "Stops after the correction, with the corrected genes as a table and sequences: no eggNOG, GFF3 or GenBank, duplications, pangenome or final_report.html; amap_report.html is still written."]); if (opt("db")) out.push([["--eggnog_db", p.db], "Builds the eggNOG database there once, and reuses it."]); return out; } - function render() { - var all = lines(); - var text = '$ nextflow run PathoGenOmics-Lab/AMAP'; + // A command as the page writes it: a span per line, with its newline in + // it, and a span per word, a flag in .f and a value in .v, so that a + // narrow screen wraps between words, under the line it continues. + // A continued line keeps its indent with its first word and its closing + // backslash with its last, so neither sits on a line of its own on a + // phone; what Copy takes is the same text. + function commandHtml(all) { + var rows = [['$ nextflow', 'run', 'PathoGenOmics-Lab/AMAP']]; all.forEach(function (line) { - var flags = line[0]; - text += " \\\n "; - text += flags - .map(function (part) { - return /^-/.test(part) ? '' + esc(part) + "" : '' + esc(part) + ""; + rows.push( + line[0].map(function (part) { + return '' + esc(part) + ""; }) - .join(" "); + ); }); - commandEl.innerHTML = text; - explainEl.innerHTML = all + return rows + .map(function (row, k) { + var parts = row.slice(); + var more = k < rows.length - 1; + if (k > 0) parts[0] = ' ' + parts[0] + ""; + if (more) parts[parts.length - 1] = '' + parts[parts.length - 1] + " \\"; + // An indented line that wraps hangs four characters further in, + // as the docs' commands do (--i is its indent). + return ' 0 ? ' style="--i:4"' : "") + ">" + parts.join(" ") + (more ? "\n" : "") + ""; + }) + .join(""); + } + + function explainHtml(all) { + return all .map(function (line) { - return "
  • " + esc(line[0].filter(function (part) { + var flags = line[0].filter(function (part) { return /^-/.test(part); - }).join(" ")) + "" + esc(line[1]) + "
    • "; + }); + return ( + '
  • ' + + flags + .map(function (flag) { + return '' + esc(flag) + ""; + }) + .join(" ") + + "" + esc(line[1]) + "
    • " + ); }) .join(""); - var db = builder.querySelector('[data-opt="db"]'); + } + + function notes() { + var out = []; + var small = opt("small"); + if (choice.engine === "slurm") { + out.push("On a cluster the slurm profile sets the ceilings, so a laptop's do not apply."); + out.push( + "Bakta's jobs ask for --qos=medium; if your cluster has none, see " + + 'HPC clusters.' + ); + } + if (choice.engine !== "docker") { + out.push( + "The singularity profile keeps the converted image in singularity_cache/ under --outdir, " + + "and that wins over NXF_SINGULARITY_CACHEDIR. To keep one shared copy, put " + + "singularity.cacheDir = '/shared/singularity_cache' in a config passed with -c." + ); + } + if (small && choice.engine === "docker") { + out.push("8 GB is what Docker Desktop usually gives: if docker info reports more, raise --max_memory to it."); + } else if (small) { + out.push("Set --max_memory to the memory the machine has free."); + } + if (small && opt("only") && onlyByLaptop) { + out.push("Only the corrected genes is ticked for you: eggNOG loads its whole database into memory and asks for 50 GB. Untick it on a machine that has that much."); + } else if (small && !opt("only")) { + out.push("eggNOG loads its whole database into memory and asks for 50 GB, which a laptop does not have: tick Only the corrected genes, or run it on a larger machine."); + } + var dbBox = box("db"); + if (dbBox && dbBox.disabled) { + out.push("Keep the eggNOG database waits for eggNOG: with Only the corrected genes the run never builds it."); + } + if (choice.organism === "other") { + out.push( + "The annotation and the genome must be the same assembly. How many proteins AMAP took from it is in the run's work folder: " + + "grep -h 'Curated proteins' work/*/*/.command.out." + ); + } + return out; + } + + function render() { + var small = box("small"); + if (small) small.disabled = choice.engine === "slurm"; + var db = box("db"); if (db) db.disabled = opt("only"); + var all = lines(); + commandEl.innerHTML = commandHtml(all); + if (checkEl) checkEl.innerHTML = commandHtml(all.concat([[["--validate_only"], ""]])); + explainEl.innerHTML = explainHtml(all); if (noteEl) { - var notes = []; - if (choice.engine === "docker" && opt("small")) { - notes.push("On macOS and Windows, Docker Desktop has its own memory limit, and it is the one that counts: keep --max_memory at or below it (Settings, Resources, Memory)."); - } - if (choice.organism === "other") { - notes.push("The annotation and the genome must be the same assembly. AMAP prints how many proteins it extracted: check it looks like a genome's worth."); - } - noteEl.innerHTML = notes.join(" "); - noteEl.hidden = !notes.length; + var list = notes(); + noteEl.innerHTML = list + .map(function (note) { + return "

    " + note + "

    "; + }) + .join(""); + noteEl.hidden = !list.length; } + // What changed, for a screen reader (announce() is the page's status + // line): each line by its first flag, so a new profile reads as one. + var now = {}; + all.forEach(function (line) { + now[line[0][0]] = line[0].join(" "); + }); + if (said) { + var parts = []; + Object.keys(now).forEach(function (flag) { + if (!(flag in said)) parts.push("adds " + now[flag]); + else if (said[flag] !== now[flag]) parts.push(now[flag]); + }); + Object.keys(said).forEach(function (flag) { + if (!(flag in now)) parts.push("drops " + said[flag]); + }); + if (parts.length && typeof announce === "function") announce("Command updated: " + parts.join("; ")); + } + said = now; } - builder.addEventListener("click", function (event) { - var button = event.target.closest("[data-choice] button"); - if (!button) return; + // Each question is a radio group: one tab stop, and the arrow keys move + // the choice along it. + function pick(button, focus) { var group = button.closest("[data-choice]"); choice[group.getAttribute("data-choice")] = button.getAttribute("data-value"); - group.querySelectorAll("button").forEach(function (b) { - b.setAttribute("aria-pressed", b === button ? "true" : "false"); + group.querySelectorAll('[role="radio"]').forEach(function (b) { + var on = b === button; + b.setAttribute("aria-checked", on ? "true" : "false"); + b.tabIndex = on ? 0 : -1; }); + if (focus) button.focus(); + var only = box("only"); + if (choice.engine === "slurm" && onlyByLaptop && only) { + only.checked = false; + onlyByLaptop = false; + } + render(); + } + + builder.addEventListener("click", function (event) { + var button = event.target.closest('[data-choice] [role="radio"]'); + if (button) pick(button, false); + }); + + builder.addEventListener("keydown", function (event) { + var button = event.target.closest('[data-choice] [role="radio"]'); + if (!button) return; + var radios = Array.prototype.slice.call(button.closest("[data-choice]").querySelectorAll('[role="radio"]')); + var at = radios.indexOf(button); + var to = { ArrowRight: at + 1, ArrowDown: at + 1, ArrowLeft: at - 1, ArrowUp: at - 1, Home: 0, End: radios.length - 1 }[event.key]; + if (to === undefined) return; + event.preventDefault(); + pick(radios[(to + radios.length) % radios.length], true); + }); + + builder.addEventListener("change", function (event) { + var only = box("only"); + if (event.target === box("small") && only) { + // A laptop cannot hold eggNOG's database: the run stops before it, + // unless the reader asks otherwise. + if (event.target.checked && !only.checked) { + only.checked = true; + onlyByLaptop = true; + } else if (!event.target.checked && onlyByLaptop) { + only.checked = false; + onlyByLaptop = false; + } + } else if (event.target === only) { + onlyByLaptop = false; + } render(); }); - builder.addEventListener("change", render); render(); }); /* โ”€โ”€ Embedded report โ”€โ”€ */ - // The report takes the mouse wheel for its own plots, so it only takes the - // mouse after a click, and gives it back when the pointer leaves. + // The example report is 4.5 MB of scripts and plots, so the page shows a + // picture of it until the reader asks for it. Once open it takes the mouse + // and the keyboard for its own plots, and gives them back when the pointer + // or focus leaves it, a touch lands outside it, Escape is pressed or Done + // is tapped. Until then it is inert: none of its links is a tab stop. document.querySelectorAll("[data-report-stage]").forEach(function (stage) { var cover = stage.querySelector(".report-cover"); var frame = stage.querySelector("iframe"); + // Done sits in the window's bar, outside the stage: a touch, a pointer + // or focus going to it does not close the report before it is tapped. + var frameBox = stage.closest(".report-frame"); + var done = frameBox ? frameBox.querySelector(".report-done") : null; + var toDone = function (target) { + return !!(done && target && done.contains(target)); + }; if (!cover || !frame) return; - cover.addEventListener("click", function () { + frame.inert = true; + + function rest(refocus) { + if (!stage.classList.contains("live")) return; + stage.classList.remove("live"); + frame.inert = true; + if (done) done.hidden = true; + if (refocus) cover.focus(); + } + + function onEscape(event) { + if (event.key === "Escape") rest(true); + } + + function open() { + if (!frame.getAttribute("src") && frame.getAttribute("data-src")) { + frame.addEventListener( + "load", + function () { + stage.classList.add("loaded"); + var label = cover.querySelector("span"); + if (label) label.textContent = "Explore the example report"; + // Escape inside the report, where the page can reach it (not + // from a file:// address, which Chrome keeps apart). + try { + frame.contentWindow.document.addEventListener("keydown", onEscape); + } catch (e) { + /* another origin: Escape works from the page only */ + } + }, + { once: true } + ); + frame.setAttribute("src", frame.getAttribute("data-src")); + } stage.classList.add("live"); + frame.inert = false; + if (done) done.hidden = false; try { frame.focus(); } catch (e) { /* nothing to focus yet */ } + } + + cover.addEventListener("click", open); + if (done) { + done.addEventListener("click", function () { + rest(true); + }); + } + stage.addEventListener("mouseleave", function (event) { + if (!toDone(event.relatedTarget)) rest(false); }); - stage.addEventListener("mouseleave", function () { - stage.classList.remove("live"); + stage.addEventListener("focusout", function (event) { + if (event.relatedTarget && !stage.contains(event.relatedTarget) && !toDone(event.relatedTarget)) rest(false); + }); + document.addEventListener( + "touchstart", + function (event) { + if (!stage.contains(event.target) && !toDone(event.target)) rest(false); + }, + { passive: true } + ); + document.addEventListener("keydown", function (event) { + if (stage.classList.contains("live")) onEscape(event); }); }); /* โ”€โ”€ Documentation: search, page contents, menu โ”€โ”€ */ - var searchBox = document.querySelector("[data-docs-search]"); - var resultsBox = document.querySelector("[data-docs-results]"); - if (searchBox && resultsBox && window.AMAP_DOCS_INDEX) { - // The index lists URLs from the docs root; this page may sit a folder down. - var indexScript = document.querySelector('script[src$="assets/search-index.js"]'); - var root = indexScript ? indexScript.getAttribute("src").replace("assets/search-index.js", "") : ""; - var active = -1; + // An old anchor (a heading renamed since, or GitHub's anchor for a + // changelog heading) leads to the heading's current one, which goes in the + // address so the contents and the arrival mark follow it. + function followAlias() { + var id = decodeURIComponent(location.hash.slice(1)); + var alias = id && document.getElementById(id); + if (!alias || !alias.classList.contains("anchor-alias")) return; + var to = alias.getAttribute("data-to"); + var target = document.getElementById(to); + if (!target) return; + try { + history.replaceState(history.state, "", "#" + to); + } catch (e) {} + target.scrollIntoView(); + } + followAlias(); + window.addEventListener("hashchange", followAlias); - var escapeHtml = function (text) { - return text.replace(/&/g, "&").replace(//g, ">").replace(/"/g, """); - }; + var header = document.querySelector("header.top"); + // The sticky header's height, its border included, as the stylesheet sets it. + var HEADER_GAP = parseFloat(getComputedStyle(document.documentElement).getPropertyValue("--header-h")) || 61; - // Words match where a word starts, so "skip" finds --skip_typing and not - // the middle of another word. - var startsWord = function (w, flags) { - return new RegExp("(^|[^a-z0-9])(" + w.replace(/[.*+?^${}()|[\]\\]/g, "\\$&") + ")", flags); - }; + function escapeHtml(text) { + return String(text).replace(/&/g, "&").replace(//g, ">").replace(/"/g, """); + } + + function escapeRe(text) { + return text.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); + } - var highlight = function (text, words) { - var safe = escapeHtml(text); - words.forEach(function (w) { - safe = safe.replace(startsWord(escapeHtml(w), "ig"), "$1$2"); + // โ”€โ”€ The index โ”€โ”€ + // One file, assets/search-index.js, serves every page. It loads the first + // time a reader reaches for search (focus, pointer, shortcut), not with the + // page; a page that already has it uses it as it is. + + var indexState = window.AMAP_DOCS_INDEX ? "ready" : "idle"; + var indexWaiting = []; + + function indexEntries() { + var entries = window.AMAP_DOCS_INDEX || []; + if (!entries.prepared) { + entries.forEach(function (e) { + e._page = (e.page || "").toLowerCase(); + e._section = (e.section || "").toLowerCase(); + e._text = (e.text || "").toLowerCase(); + e._code = (e.code || "").toLowerCase(); + e._base = (e.url || "").split("#")[0]; + e._kind = e.kind || (/^about\/changelog/.test(e.url) ? "changelog" : e.section ? "section" : "page"); + }); + entries.prepared = true; + } + return entries; + } + + function loadIndex(then) { + if (window.AMAP_DOCS_INDEX) indexState = "ready"; + if (indexState === "ready" || indexState === "failed") { + if (then) then(); + return; + } + if (then && indexWaiting.indexOf(then) < 0) indexWaiting.push(then); + if (indexState === "loading") return; + indexState = "loading"; + var script = document.createElement("script"); + script.src = ROOT + "assets/search-index.js"; + script.async = true; + var settle = function () { + indexState = window.AMAP_DOCS_INDEX ? "ready" : "failed"; + var waiting = indexWaiting; + indexWaiting = []; + waiting.forEach(function (fn) { + fn(); }); - return safe; }; + script.onload = settle; + script.onerror = settle; + document.head.appendChild(script); + } + + // โ”€โ”€ Matching and ranking โ”€โ”€ - var snippet = function (text, word) { - var at = text.toLowerCase().indexOf(word); - if (at < 0) return text.slice(0, 140); - var start = Math.max(0, at - 60); - return (start > 0 ? "โ€ฆ" : "") + text.slice(start, start + 150) + (start + 150 < text.length ? "โ€ฆ" : ""); + // Words people type for what the docs call something else. + var SYNONYMS = { + oom: ["memory"], + db: ["database"], + cluster: ["hpc", "slurm"], + mamba: ["conda"], + gbk: ["genbank"], + vanish: ["missing", "ignored"], + vanished: ["missing", "ignored"], + disappear: ["missing", "ignored"], + disappeared: ["missing", "ignored"], + dropped: ["missing", "ignored"], + proteome: ["faa", "proteins"], + laptop: ["docker desktop"] + }; + + // Words a question in plain words is made of and every page has: "how to + // run on slurm" searches for run and slurm. A query of nothing else keeps + // them. + var STOP = {}; + ("a an the to of on in is are be do does did how what where why which when who can i my me we you use using " + + "much many mean means get with for it this that there").split(" ").forEach(function (w) { + STOP[w] = true; + }); + // Words in nearly every sentence: never searched for, even alone ("the" + // would list every page and mark it inside "them"). + var NOISE = {}; + "a an the and or of to in on for is are be it this that with by at as".split(" ").forEach(function (w) { + NOISE[w] = true; + }); + + // A query word matches where a word starts, so "skip" finds --skip_typing and + // not the middle of another word. A lone number matches only as a whole + // number ("case 4", not 40). A long word also matches inside other words + // ("mastertable" in AMAPmastertable.tsv). + function term(word, weight) { + var core = escapeRe(word); + var tail = /^[0-9]+$/.test(word) ? "(?![0-9]|[.,][0-9])" : ""; + return { + word: word, + weight: weight, + start: new RegExp("(^|[^a-z0-9])(" + core + ")" + tail, "g"), + inside: word.length >= 5 ? new RegExp("()(" + core + ")", "g") : null }; + } - var search = function (query) { - var words = query.toLowerCase().split(/\s+/).filter(function (w) { - return w.length > 1; + // The words as typed, cleaned of punctuation. + function typedWords(query) { + return query + .toLowerCase() + .split(/\s+/) + .map(function (w) { + return w.replace(/^[^a-z0-9-]+|[^a-z0-9]+$/g, ""); + }) + .filter(function (w) { + return w.length > 1 || /^[0-9]$/.test(w); }); - if (!words.length) return [words, []]; - var hits = []; - window.AMAP_DOCS_INDEX.forEach(function (entry) { - var page = entry.page.toLowerCase(); - var section = entry.section.toLowerCase(); - var text = entry.text.toLowerCase(); - var score = 0; - for (var k = 0; k < words.length; k++) { - var pattern = startsWord(words[k], "i"); - var inSection = pattern.test(section); - var inPage = pattern.test(page); - var inText = pattern.test(text); - if (!inSection && !inPage && !inText) return; - score += (inSection ? 5 : 0) + (inPage ? 3 : 0) + (inText ? 1 : 0); + } + + // The words searched for: no noise word, and no question word when the + // query has a word that names something. + function queryWords(query) { + var words = typedWords(query).filter(function (w) { + return !NOISE[w]; + }); + var content = words.filter(function (w) { + return !STOP[w]; + }); + return content.length ? content : words; + } + + function termGroups(words) { + return words.map(function (w) { + return [term(w, 1)].concat( + (SYNONYMS[w] || []).map(function (s) { + return term(s, 0.8); + }) + ); + }); + } + + function count(re, text) { + if (!text) return 0; + re.lastIndex = 0; + var n = 0; + while (re.exec(text) && n < 60) n++; + return n; + } + + function hits(t, text) { + var n = count(t.start, text); + return n || !t.inside ? n : count(t.inside, text); + } + + // One query word against one entry: the best of the word and its synonyms. + function scoreTerm(group, e) { + var best = null; + group.forEach(function (t) { + var page = hits(t, e._page); + var section = hits(t, e._section); + var text = hits(t, e._text); + var code = hits(t, e._code); + if (!page && !section && !text && !code) return; + var s = + 1 + + (page ? 5 : 0) + + (section ? 3 : 0) + + (e._section === t.word || e._page === t.word ? 3 : 0) + + Math.log(1 + text) + + 0.5 * Math.log(1 + code); + s *= t.weight; + if (!best || s > best.score) best = { score: s, tf: page + section + text + code }; + }); + return best; + } + + // Words typed next to each other score more where they stand together. + function phraseBonus(words, e) { + var bonus = 0; + for (var k = 0; k + 1 < words.length; k++) { + var tail = /^[0-9]+$/.test(words[k + 1]) ? "(?![0-9]|[.,][0-9])" : ""; + var re = new RegExp("(^|[^a-z0-9])" + escapeRe(words[k]) + "[^a-z0-9]{1,3}" + escapeRe(words[k + 1]) + tail, "g"); + if (count(re, e._section)) bonus += 4; + var inText = count(re, e._text); + if (inText) bonus += 2 + Math.log(1 + inText); + } + return bonus; + } + + var vocabulary = null; + + function buildVocabulary() { + if (vocabulary) return vocabulary; + vocabulary = {}; + indexEntries().forEach(function (e) { + (e._page + " " + e._section + " " + e._text).replace(/[a-z][a-z0-9]{2,}/g, function (w) { + vocabulary[w] = (vocabulary[w] || 0) + 1; + return w; + }); + }); + return vocabulary; + } + + // Edits between two words, a swap of neighbours counting as one. + function distance(a, b, limit) { + if (Math.abs(a.length - b.length) > limit) return limit + 1; + var prev2 = null; + var prev = []; + for (var j = 0; j <= b.length; j++) prev.push(j); + for (var i = 1; i <= a.length; i++) { + var row = [i]; + var low = i; + for (var k = 1; k <= b.length; k++) { + var cost = a.charAt(i - 1) === b.charAt(k - 1) ? 0 : 1; + var v = Math.min(prev[k] + 1, row[k - 1] + 1, prev[k - 1] + cost); + if (prev2 && i > 1 && k > 1 && a.charAt(i - 1) === b.charAt(k - 2) && a.charAt(i - 2) === b.charAt(k - 1)) { + v = Math.min(v, prev2[k - 2] + 1); + } + row.push(v); + if (v < low) low = v; + } + if (low > limit) return limit + 1; + prev2 = prev; + prev = row; + } + return prev[b.length]; + } + + // With nothing found: the nearest words the docs do use. + function suggestion(words) { + var vocab = buildVocabulary(); + var changed = false; + var fixed = words.map(function (w) { + var group = termGroups([w])[0]; + var known = Object.keys(vocab).some(function (v) { + return group.some(function (t) { + return v.indexOf(t.word) === 0 || (t.inside && v.indexOf(t.word) >= 0); + }); + }); + if (known || w.length < 4 || !/^[a-z]+$/.test(w)) return w; + var limit = w.length <= 5 ? 1 : 2; + var best = null; + var bestD = limit + 1; + Object.keys(vocab).forEach(function (v) { + var d = distance(w, v, limit); + if (d < bestD || (d === bestD && best && vocab[v] > vocab[best])) { + best = v; + bestD = d; } - hits.push({ entry: entry, score: score }); }); - hits.sort(function (a, b) { + if (best && bestD <= limit) { + changed = true; + return best; + } + return w; + }); + return changed ? fixed.join(" ") : null; + } + + // Every entry that has every word, grouped by page. A page ranks by its best + // entry plus how much of the page is about the words; Changelog entries + // count half, so the page that explains a thing comes before the release + // that changed it. When no entry has every word of three or more, the + // entries that miss one word come instead, scored by how many they have. + function searchDocs(query) { + var words = queryWords(query); + var result = { words: words, groups: [], total: 0, suggestion: null, partial: false }; + if (!words.length) return result; + var groupsOfTerms = termGroups(words); + var byPage = {}; + var order = []; + var entries = indexEntries(); + // Each word against each entry, once. A word most entries have ("run") + // weighs less than one few have ("slurm"), so in a question the word + // that names the subject decides. + var per = entries.map(function (e) { + return groupsOfTerms.map(function (group) { + return scoreTerm(group, e); + }); + }); + var weights = groupsOfTerms.map(function (_g, k) { + var df = 0; + per.forEach(function (row) { + if (row[k]) df++; + }); + return Math.log(1 + entries.length / (1 + df)); + }); + var mean = weights.reduce(function (a, b) { + return a + b; + }, 0) / weights.length; + var allowMissing = 0; + var matched = function (row) { + var score = 0; + var tf = 0; + var missing = 0; + for (var k = 0; k < row.length; k++) { + var s = row[k]; + if (!s) { + if (++missing > allowMissing) return null; + continue; + } + score += s.score * (mean ? weights[k] / mean : 1); + tf += s.tf; + } + score *= (row.length - missing) / row.length; + return { score: score, tf: tf }; + }; + var matches = per.map(matched); + if (words.length >= 3 && !matches.some(Boolean)) { + allowMissing = 1; + result.partial = true; + matches = per.map(matched); + } + entries.forEach(function (e, i) { + var m = matches[i]; + if (!m) return; + var score = m.score; + var tf = m.tf; + score += phraseBonus(words, e); + if (e._kind === "changelog") score *= 0.5; + var key = e._base; + var g = byPage[key]; + if (!g) { + g = byPage[key] = { page: e.page, url: e._base, top: null, sections: [], best: 0, tf: 0 }; + order.push(g); + } + var hit = { entry: e, score: score }; + if (!e.section && e.url === e._base) g.top = hit; + else g.sections.push(hit); + g.best = Math.max(g.best, score); + g.tf += tf; + result.total++; + }); + order.forEach(function (g) { + g.score = g.best + Math.log(1 + g.tf) * (g.top || g.sections.length ? 1 : 0); + if (/^about\/changelog/.test(g.url)) g.score *= 0.6; + g.sections.sort(function (a, b) { return b.score - a.score; }); - return [words, hits.slice(0, 8)]; + }); + order.sort(function (a, b) { + return b.score - a.score; + }); + result.groups = order; + if (!result.total) result.suggestion = suggestion(words); + return result; + } + + // Where the words are in a piece of text, as [start, end) ranges. + function ranges(text, words) { + var lower = text.toLowerCase(); + var found = []; + termGroups(words).forEach(function (group) { + group.forEach(function (t) { + [t.start, t.inside].forEach(function (re) { + if (!re) return; + re.lastIndex = 0; + var m; + var n = 0; + while ((m = re.exec(lower)) && n++ < 40) { + var at = m.index + m[1].length; + found.push([at, at + m[2].length]); + } + }); + }); + }); + found.sort(function (a, b) { + return a[0] - b[0]; + }); + var merged = []; + found.forEach(function (r) { + var last = merged[merged.length - 1]; + if (last && r[0] <= last[1]) last[1] = Math.max(last[1], r[1]); + else merged.push(r.slice()); + }); + return merged; + } + + // The text with the words marked, and each flag (--max_memory, -profile) + // held on one line, so a narrow list never leaves its dashes at the end of + // one line and its name on the next. + function highlight(text, words) { + var marks = ranges(text, words); + var flags = []; + var re = /(^|[^\w-])(--?[A-Za-z][\w-]*)/g; + var m; + while ((m = re.exec(text))) { + var from = m.index + m[1].length; + flags.push([from, from + m[2].length]); + } + var cuts = [0, text.length]; + marks.concat(flags).forEach(function (r) { + cuts.push(r[0], r[1]); + }); + cuts = cuts + .sort(function (a, b) { + return a - b; + }) + .filter(function (c, i, all) { + return i === 0 || c !== all[i - 1]; + }); + var inside = function (list, at) { + return list.some(function (r) { + return r[0] <= at && at < r[1]; + }); }; + var out = ""; + var inFlag = false; + for (var i = 0; i + 1 < cuts.length; i++) { + var a = cuts[i]; + var piece = escapeHtml(text.slice(a, cuts[i + 1])); + var flag = inside(flags, a); + if (flag !== inFlag) { + out += flag ? '' : ""; + inFlag = flag; + } + out += inside(marks, a) ? "" + piece + "" : piece; + } + return out + (inFlag ? "" : ""); + } + + // A line of prose around the first place the words appear, from the + // section's text and never from its code. + function snippet(text, words) { + if (!text) return ""; + var found = ranges(text, words); + var at = found.length ? found[0][0] : 0; + var start = 0; + if (at > 70) { + var sentence = Math.max(text.lastIndexOf(". ", at), text.lastIndexOf("? ", at), text.lastIndexOf(": ", at)); + start = sentence >= 0 && at - sentence < 110 ? sentence + 2 : text.indexOf(" ", at - 60) + 1; + } + var end = start + 170; + if (end < text.length) end = text.lastIndexOf(" ", end) > start ? text.lastIndexOf(" ", end) : end; + else end = text.length; + // A cut snippet ends in its ellipsis alone, never in ".โ€ฆ" or ",โ€ฆ". + var body = text.slice(start, end); + if (end < text.length) body = body.replace(/[\s.,;:]+$/, "") + "โ€ฆ"; + return (start > 0 ? "โ€ฆ" : "") + body; + } + + // โ”€โ”€ The search box: an ARIA combobox โ”€โ”€ + // Focus stays in the box; Up and Down move through the results, Enter opens + // one, Esc closes. The panel closes when focus leaves the search. + + var searchBoxes = []; + var HINT = 'Try another word, or a flag such as --skip_typing.'; - var renderResults = function () { - var found = search(searchBox.value.trim()); - var words = found[0]; - var hits = found[1]; + function bindSearch(input, panel, options) { + var box = { input: input, panel: panel }; + var listId = panel.id || (panel.id = options.id); + var container = options.container || input.parentElement; + var list = []; + var active = -1; + var words = []; + + input.setAttribute("role", "combobox"); + input.setAttribute("aria-expanded", "false"); + input.setAttribute("aria-controls", listId); + input.setAttribute("aria-autocomplete", "list"); + input.setAttribute("autocomplete", "off"); + input.setAttribute("spellcheck", "false"); + input.setAttribute("enterkeyhint", "go"); + panel.setAttribute("role", "listbox"); + panel.setAttribute("aria-label", "Search results"); + // A list that scrolls is a Tab stop of its own in some browsers; this one + // is reached with the arrows, so Tab goes on past it and closes it. + panel.setAttribute("tabindex", "-1"); + + function show() { + panel.hidden = false; + input.setAttribute("aria-expanded", "true"); + if (options.onToggle) options.onToggle(true); + } + + function hide() { + panel.hidden = true; + input.setAttribute("aria-expanded", "false"); + input.removeAttribute("aria-activedescendant"); active = -1; - if (!words.length) { - resultsBox.hidden = true; + if (options.onToggle) options.onToggle(false); + } + + function select(k) { + active = k; + list.forEach(function (option, j) { + var on = j === k; + option.classList.toggle("active", on); + option.setAttribute("aria-selected", on ? "true" : "false"); + }); + if (k < 0 || !list[k]) { + input.removeAttribute("aria-activedescendant"); return; } - resultsBox.hidden = false; - if (!hits.length) { - resultsBox.innerHTML = '
    Nothing matches. Try another word.
    '; - return; + input.setAttribute("aria-activedescendant", list[k].id); + var scroller = panel.scrollHeight > panel.clientHeight ? panel : options.scroller; + if (scroller) { + var o = list[k].getBoundingClientRect(); + var s = scroller.getBoundingClientRect(); + var pad = 36; + if (o.top < s.top + pad) scroller.scrollTop -= s.top + pad - o.top; + else if (o.bottom > s.bottom - pad) scroller.scrollTop += o.bottom - (s.bottom - pad); } - resultsBox.innerHTML = hits - .map(function (hit) { - var e = hit.entry; - var title = e.section ? e.page + " โ€บ " + e.section : e.page; - return ( - '' + highlight(title, words) + "" + - highlight(snippet(e.text, words[0]), words) + "" + } + + function option(k, cls, href, inner, extra) { + return ( + '" + inner + "" + ); + } + + function render(query) { + var found = searchDocs(query); + words = found.words; + var html = ""; + var k = 0; + var best = -1; + var bestScore = -1; + if (!found.total) { + html += ''; + if (found.suggestion) { + html += option(k++, "r-suggest", "#", "Did you mean " + escapeHtml(found.suggestion) + "?", ' data-suggest="' + escapeHtml(found.suggestion) + '"'); + } else { + html += '"; + } + // Never a dead end: Enter, or a tap, still has somewhere to go. + html += option(k++, "r-page", ROOT + "documentation.html", 'All the docsEvery page, by section'); + announce("Nothing matches " + query + (found.suggestion ? ". Did you mean " + found.suggestion + "?" : ""), 350); + } else { + var groups = found.groups.slice(0, 8); + var pages = found.groups.length; + var line = + (found.partial ? "No page has every word: " : "") + + found.total + (found.total === 1 ? " result" : " results") + (found.partial ? " with all but one word," : "") + + " in " + pages + (pages === 1 ? " page" : " pages"); + html += '"; + groups.forEach(function (g, gi) { + var more = g.sections.length; + var meta = more ? more + (more === 1 ? " section" : " sections") : ""; + var top = g.top ? snippet(g.top.entry.text, words) : ""; + // Enter opens the best match of the best page. + if (gi === 0 && g.top && g.top.score > bestScore) { + best = k; + bestScore = g.top.score; + } + html += option( + k++, + "r-page", + ROOT + g.url, + '' + highlight(g.page, words) + "" + + (meta ? '' + meta + "" : "") + + (top ? "" + highlight(top, words) + "" : "") ); - }) - .join(""); - }; + g.sections.slice(0, 2).forEach(function (hit) { + if (gi === 0 && hit.score > bestScore) { + best = k; + bestScore = hit.score; + } + var e = hit.entry; + html += option( + k++, + "r-sec", + ROOT + e.url, + '' + highlight(e.section, words) + "" + + "" + highlight(snippet(e.text, words), words) + "" + ); + }); + }); + announce( + (found.partial ? "No page has every word. " : "") + + found.total + (found.total === 1 ? " result" : " results") + (found.partial ? " with all but one word" : "") + + " for " + query + ", in " + pages + (pages === 1 ? " page" : " pages"), + 350 + ); + } + html += foot(k); + panel.innerHTML = html; + list = Array.prototype.slice.call(panel.querySelectorAll('[role="option"]')); + panel.scrollTop = 0; + show(); + // The match that Enter opens is marked from the start. + select(found.total ? best : 0); + } - var move = function (step) { - var links = resultsBox.querySelectorAll("a"); - if (!links.length) return; - active = (active + step + links.length) % links.length; - links.forEach(function (link, k) { - link.classList.toggle("active", k === active); - }); - links[active].scrollIntoView({ block: "nearest" }); - }; + // The keys under the list: the arrows and Enter only when there is + // something to move to and open. + function foot(options) { + return options + ? '' + : ''; + } - searchBox.addEventListener("input", renderResults); - searchBox.addEventListener("keydown", function (event) { - if (event.key === "ArrowDown") { + function run() { + var query = input.value.trim(); + if (options.onQuery) options.onQuery(query); + if (!queryWords(query).length) { + list = []; + if (!typedWords(query).length) { + hide(); + return; + } + // Only words such as "the", which every page has: a hint, no list. + input.removeAttribute("aria-activedescendant"); + active = -1; + panel.innerHTML = '" + foot(0); + show(); + announce("Try another word, or a flag such as --skip_typing.", 350); + return; + } + if (indexState !== "ready") { + loadIndex(run); + if (indexState !== "ready") { + panel.innerHTML = + '
    ' + (indexState === "failed" ? "The search index did not load. Reload the page to try again." : "Loading the search indexโ€ฆ") + "
    "; + list = []; + show(); + return; + } + } + render(query); + } + + function choose(link, viaKey) { + if (link.hasAttribute("data-suggest")) { + input.value = link.getAttribute("data-suggest"); + run(); + input.focus(); + return false; + } + sessionSet("amap.searchHit", { url: link.href, words: words }); + var samePage = link.href.split("#")[0] === location.href.split("#")[0]; + if (samePage) { + hide(); + if (options.onChoose) options.onChoose(); + } + if (viaKey) location.href = link.href; + return true; + } + + input.addEventListener("input", run); + input.addEventListener("focus", function () { + loadIndex(); + if (input.value.trim() && panel.hidden) run(); + }); + container.addEventListener("pointerenter", function () { + loadIndex(); + }); + input.addEventListener("keydown", function (event) { + if (event.key === "ArrowDown" || event.key === "ArrowUp") { event.preventDefault(); - move(1); - } else if (event.key === "ArrowUp") { + if (panel.hidden) run(); + if (!list.length) return; + var step = event.key === "ArrowDown" ? 1 : -1; + select(active < 0 ? (step > 0 ? 0 : list.length - 1) : (active + step + list.length) % list.length); + } else if (event.key === "Home" && event.altKey) { event.preventDefault(); - move(-1); + if (list.length) select(0); } else if (event.key === "Enter") { - var links = resultsBox.querySelectorAll("a"); - var pick = links[active >= 0 ? active : 0]; - if (pick) window.location.href = pick.getAttribute("href"); - } else if (event.key === "Escape") { - searchBox.value = ""; - resultsBox.hidden = true; + var pick = list[active >= 0 ? active : 0]; + if (pick && !panel.hidden) { + event.preventDefault(); + choose(pick, true); + } + } else if (event.key === "Escape" && !options.sheet) { + if (!panel.hidden) { + event.preventDefault(); + hide(); + } else if (input.value) { + event.preventDefault(); + input.value = ""; + } } }); + // A press inside the panel keeps focus in the box, so the panel is still + // there when the click lands. + panel.addEventListener("mousedown", function (event) { + event.preventDefault(); + }); + panel.addEventListener("click", function (event) { + var link = event.target.closest('[role="option"]'); + if (!link) return; + if (!choose(link, false)) event.preventDefault(); + }); + panel.addEventListener("mousemove", function (event) { + var link = event.target.closest('[role="option"]'); + if (link && list.indexOf(link) !== active) select(list.indexOf(link)); + }); + container.addEventListener("focusout", function (event) { + if (!event.relatedTarget || !container.contains(event.relatedTarget)) hide(); + }); document.addEventListener("click", function (event) { - if (!event.target.closest(".docs-search")) resultsBox.hidden = true; + // The header's Search button hands focus to this box; its click is not + // a click away. + if (!container.contains(event.target) && !event.target.closest(".search-btn")) hide(); }); - // "/" jumps to the search box, as on most documentation sites. - document.addEventListener("keydown", function (event) { - if (event.key === "/" && !/input|textarea|select/i.test(document.activeElement.tagName)) { + + box.run = run; + box.hide = hide; + searchBoxes.push(box); + return box; + } + + // The sidebar box of the documentation pages. + var sideInput = document.querySelector("[data-docs-search]"); + var sidePanel = document.querySelector("[data-docs-results]"); + if (sideInput && sidePanel) { + var key = document.createElement("kbd"); + key.className = "search-key"; + key.setAttribute("aria-hidden", "true"); + key.textContent = "/"; + sideInput.insertAdjacentElement("afterend", key); + sideInput.setAttribute("aria-keyshortcuts", "/ Control+K Meta+K"); + bindSearch(sideInput, sidePanel, { id: "docs-results", container: sideInput.closest(".docs-search") }); + } + + // โ”€โ”€ Search from every page: a button in the header, and a sheet under it โ”€โ”€ + + function pageName() { + var title = document.title.split(" | ")[0]; + if (/^AMAP:/.test(title)) return "Front page"; + if (/^How AMAP decides/.test(title)) return "How it works, illustrated"; + return title; + } + + // The page read before this one, for the sheet's "Last visited". + var here = { title: pageName(), url: location.href.split("#")[0] }; + var trail = (sessionGet("amap.trail") || []).filter(function (p) { + return p && p.url && p.url !== here.url; + }); + var lastVisited = trail[0] || null; + if (here.title !== "Page not found") sessionSet("amap.trail", [here].concat(trail).slice(0, 5)); + + var GLASS = + ''; + var QUICK = [ + ["user-guide/parameters.html", "Parameters", "Every option and its default"], + ["user-guide/outputs.html", "Outputs", "Every file a run writes"], + ["deployment/troubleshooting.html", "Troubleshooting", "When a run stops or looks wrong"], + ["getting-started/tutorial.html", "Tutorial", "One genome, start to finish"], + ["how-it-works.html", "How it works", "The correction, step by step"] + ]; + + var nav = document.querySelector(".top .nav"); + var searchButton = null; + var sheet = null; + var sheetBox = null; + var sheetOpener = null; + var scrim = null; + + function sidebarVisible() { + return !!(sideInput && window.matchMedia && window.matchMedia("(min-width: 861px)").matches && sideInput.getClientRects().length); + } + + function openSearch(opener) { + loadIndex(); + if (sidebarVisible()) { + sideInput.focus(); + sideInput.select(); + return; + } + if (sheet) openSheet(opener); + } + + if (nav && header) { + searchButton = document.createElement("button"); + searchButton.type = "button"; + searchButton.className = "search-btn"; + searchButton.setAttribute("aria-label", "Search the documentation"); + searchButton.setAttribute("aria-keyshortcuts", "/ Control+K Meta+K"); + searchButton.setAttribute("aria-expanded", "false"); + searchButton.setAttribute("aria-controls", "search-sheet"); + searchButton.innerHTML = GLASS + 'Search'; + var before = nav.querySelector(".gh-link") || nav.querySelector("[data-theme-toggle]"); + nav.insertBefore(searchButton, before); + searchButton.addEventListener("pointerenter", function () { + loadIndex(); + }); + searchButton.addEventListener("click", function () { + if (sheet && !sheet.hidden) closeSheet(true); + else openSearch(searchButton); + }); + + // Where to go from here: never the page this is, and a page once, on + // its first card. + var cards = QUICK.map(function (q) { + return [ROOT + q[0], q[1], q[2], ""]; + }); + if (lastVisited) cards.push([lastVisited.url, lastVisited.title, "Last visited", "sheet-last"]); + cards.push([ROOT + "documentation.html", "All the docs", "Every page, by section", ""]); + var listed = {}; + listed[here.url] = true; + var quick = []; + cards.forEach(function (c) { + var url = new URL(c[0], location.href).href.split("#")[0]; + if (listed[url]) return; + listed[url] = true; + quick.push( + "
  • ' + escapeHtml(c[1]) + + "" + escapeHtml(c[2]) + "
    • " + ); + }); + + sheet = document.createElement("div"); + sheet.className = "search-sheet"; + sheet.id = "search-sheet"; + sheet.hidden = true; + sheet.setAttribute("role", "dialog"); + sheet.setAttribute("aria-modal", "true"); + sheet.setAttribute("aria-label", "Search the documentation"); + sheet.innerHTML = + '
    ' + + '
    ' + GLASS + + '' + + '' + + '' + + "
    " + + '
    ' + + '' + + '" + + "
    " + + "
    "; + scrim = document.createElement("div"); + scrim.className = "search-scrim"; + scrim.hidden = true; + header.insertAdjacentElement("afterend", sheet); + header.insertAdjacentElement("afterend", scrim); + + var sheetInput = sheet.querySelector("input"); + sheetBox = bindSearch(sheetInput, sheet.querySelector(".docs-results"), { + id: "sheet-results", + container: sheet, + scroller: sheet, + sheet: true, + onQuery: function (query) { + sheet.classList.toggle("has-query", queryWords(query).length > 0); + }, + onChoose: function () { + closeSheet(false); + } + }); + sheet.querySelector(".sheet-close").addEventListener("click", function () { + closeSheet(true); + }); + sheet.addEventListener("keydown", function (event) { + if (event.key === "Escape") { event.preventDefault(); - searchBox.focus(); + closeSheet(true); } }); + sheet.addEventListener("focusout", function (event) { + var to = event.relatedTarget; + if (to && !sheet.contains(to) && to !== sheetOpener) closeSheet(false); + }); + sheet.addEventListener("click", function (event) { + var link = event.target.closest(".sheet-empty a"); + if (link && link.href.split("#")[0] === location.href.split("#")[0]) closeSheet(false); + }); + scrim.addEventListener("click", function () { + closeSheet(true); + }); + document.addEventListener("click", function (event) { + if (!sheet.hidden && !sheet.contains(event.target) && !event.target.closest(".search-btn, .here-bar")) closeSheet(false); + }); } - // The page contents follow the reader down the page. - var tocLinks = Array.prototype.slice.call(document.querySelectorAll(".docs-toc a")); - if (tocLinks.length) { - var targets = tocLinks - .map(function (link) { - return document.getElementById(link.getAttribute("href").slice(1)); - }) - .filter(Boolean); + function openSheet(opener) { + if (!sheet) return; + // Already open (Ctrl+K again): back to its box, and Esc still returns + // to what opened it first. + if (!sheet.hidden) { + var again = sheet.querySelector("input"); + again.focus(); + again.select(); + return; + } + sheetOpener = opener || document.activeElement; + sheet.hidden = false; + scrim.hidden = false; + [searchButton, hereBar].forEach(function (b) { + if (b) b.setAttribute("aria-expanded", "true"); + }); + document.documentElement.classList.add("sheet-open"); + // The page under the sheet is out of reach while it is open; the header + // stays, and focus that goes there closes the sheet. + Array.prototype.forEach.call(document.body.children, function (el) { + if (el === sheet || el === scrim || el === header || el === hereBar || el === statusLine || el.inert || /^(SCRIPT|STYLE)$/.test(el.tagName)) return; + el.inert = true; + el.setAttribute("data-sheet-inert", ""); + }); + sheet.scrollTop = 0; + // From the bar that names the section, the contents come first, with no + // keyboard popping up over them; from anywhere else, the search box. + // An earlier search left in the box would hide the contents, so the bar + // clears it. + var on = sheet.querySelector(".sheet-toc:not([hidden]) a.active"); + if (opener && opener === hereBar && on) { + var old = sheet.querySelector("input"); + if (old.value) { + old.value = ""; + sheetBox.run(); + } + on.focus({ preventScroll: true }); + var top = on.getBoundingClientRect().top - sheet.getBoundingClientRect().top; + if (top > sheet.clientHeight - 60) sheet.scrollTop = top - sheet.clientHeight / 2; + return; + } + var input = sheet.querySelector("input"); + input.focus(); + input.select(); + } + + function closeSheet(restore) { + if (!sheet || sheet.hidden) return; + sheet.hidden = true; + scrim.hidden = true; + sheetBox.hide(); + [searchButton, hereBar].forEach(function (b) { + if (b) b.setAttribute("aria-expanded", "false"); + }); + document.documentElement.classList.remove("sheet-open"); + document.querySelectorAll("[data-sheet-inert]").forEach(function (el) { + el.inert = false; + el.removeAttribute("data-sheet-inert"); + }); + if (restore && sheetOpener && sheetOpener.focus) sheetOpener.focus(); + } + + // "/" and Ctrl or Cmd+K open the search from anywhere on the site. A key + // of a single character can be turned off (WCAG 2.1.4), from the footer of + // every page; this browser remembers it. + var slashOn = true; + try { + slashOn = localStorage.getItem("amap.slash") !== "off"; + } catch (e) {} + var slashToggle = null; + function setSlash(on, save) { + slashOn = on; + if (save) { + try { + if (on) localStorage.removeItem("amap.slash"); + else localStorage.setItem("amap.slash", "off"); + } catch (e) {} + } + document.documentElement.classList.toggle("no-slash", !on); + document.querySelectorAll("[aria-keyshortcuts]").forEach(function (el) { + el.setAttribute("aria-keyshortcuts", (on ? "/ " : "") + "Control+K Meta+K"); + }); + if (slashToggle) { + // The key in the keycap style it has everywhere else on the site. + var keycap = document.createElement("kbd"); + keycap.textContent = "/"; + slashToggle.textContent = "The "; + slashToggle.appendChild(keycap); + slashToggle.appendChild(document.createTextNode(on ? " key opens the search: turn it off" : " key does nothing: turn it on")); + } + } + var foot = document.querySelector("footer.foot"); + var footBase = foot && foot.querySelector(".foot-base"); + if (foot && !footBase) { + footBase = document.createElement("div"); + footBase.className = "wrap foot-base"; + foot.appendChild(footBase); + } + if (footBase && searchButton) { + slashToggle = document.createElement("button"); + slashToggle.type = "button"; + slashToggle.className = "slash-toggle"; + slashToggle.addEventListener("click", function () { + setSlash(!slashOn, true); + announce(slashOn ? "The / key opens the search" : "The / key no longer opens the search"); + }); + footBase.appendChild(slashToggle); + } + setSlash(slashOn, false); + // A key pressed away from a text field means a real keyboard (a phone's + // own keyboard only types into fields): the footer then offers the / + // switch, which site.css hides on a touch screen until then. + document.addEventListener("keydown", function onKey(event) { + var t = event.target; + if (t && (/^(input|textarea|select)$/i.test(t.tagName) || t.isContentEditable)) return; + document.documentElement.classList.add("keys"); + document.removeEventListener("keydown", onKey); + }); + + document.addEventListener("keydown", function (event) { + var t = event.target; + var typing = t && (/^(input|textarea|select)$/i.test(t.tagName) || t.isContentEditable); + var slash = slashOn && event.key === "/" && !typing && !event.ctrlKey && !event.metaKey && !event.altKey; + var k = (event.key === "k" || event.key === "K") && (event.ctrlKey || event.metaKey) && !event.altKey && !event.shiftKey; + if (!slash && !k) return; + event.preventDefault(); + openSearch(document.activeElement && document.activeElement !== document.body ? document.activeElement : searchButton); + }); + + // โ”€โ”€ The header keeps to one row โ”€โ”€ + // The stylesheet drops header items at widths measured with the usual + // fonts. A wider font, larger text or extra letter spacing can still run + // the row over, so the header then sheds the same items, one step at a + // time, until it fits (the squeeze-1 to squeeze-6 rules in site.css). + var SQUEEZE_STEPS = 6; + function fitHeader() { + for (var s = 1; s <= SQUEEZE_STEPS; s++) header.classList.remove("squeeze-" + s); + for (var step = 1; step <= SQUEEZE_STEPS && nav.scrollWidth > nav.clientWidth + 1; step++) { + header.classList.add("squeeze-" + step); + } + } + + if (nav && header) { + fitHeader(); + var fitQueued = false; + window.addEventListener("resize", function () { + if (fitQueued) return; + fitQueued = true; + requestAnimationFrame(function () { + fitQueued = false; + fitHeader(); + }); + }); + } + + // โ”€โ”€ The page contents follow the reader down the page โ”€โ”€ + // The contents in the sidebar (docs), the box under the h1 (docs on a + // phone) and the rail of chapters (front page) all mark the section being + // read; the first of them for a heading decides its level. + + var tocLinks = Array.prototype.slice.call(document.querySelectorAll(".page-toc a, .toc-inline a, .rail a")); + var tocTargets = []; + var seenTarget = {}; + tocLinks.forEach(function (link) { + var id = decodeURIComponent((link.getAttribute("href") || "").slice(1)); + var el = id && !seenTarget[id] ? document.getElementById(id) : null; + if (!el) return; + seenTarget[id] = true; + var text = link.textContent.trim(); + // The rail's names are cut short to fit the margin; the bar and the + // sheet have room for the name the chapter gives itself, its kicker. + var kicker = link.closest(".rail") ? el.querySelector(".section-head .kicker") : null; + if (kicker) { + var number = link.querySelector(".rail-n"); + text = (number ? number.textContent.trim() + " " : "") + kicker.textContent.trim(); + } + tocTargets.push({ id: id, el: el, text: text, level: link.closest(".toc-3") ? 3 : 2 }); + }); + + var hereBar = null; + var hereLabel = null; + var hereProgress = null; + var currentId = null; + + if (tocTargets.length) { + // The sheet lists this page's sections, so a phone has its contents one + // tap away (from the header's search, or the bar that names the section). + if (sheet) { + var sheetToc = sheet.querySelector(".sheet-toc"); + sheetToc.querySelector("ol").innerHTML = tocTargets + .filter(function (t) { + return t.level === 2; + }) + .map(function (t) { + return '
  • ' + escapeHtml(t.text) + "
    • "; + }) + .join(""); + sheetToc.hidden = false; + Array.prototype.push.apply(tocLinks, sheetToc.querySelectorAll("a")); + } + var main = document.querySelector("main"); + if (main && header) { + hereBar = document.createElement("button"); + hereBar.type = "button"; + hereBar.className = "here-bar"; + hereBar.setAttribute("aria-controls", "search-sheet"); + hereBar.setAttribute("aria-expanded", "false"); + hereBar.innerHTML = + '' + + '' + + ''; + hereLabel = hereBar.querySelector(".here-label"); + hereProgress = hereBar.querySelector(".here-progress"); + (sheet || header).insertAdjacentElement("afterend", hereBar); + document.documentElement.classList.add("has-here-bar"); + hereBar.addEventListener("click", function () { + openSheet(hereBar); + }); + } + } + + // Where an anchored heading lands: the page's scroll padding plus the + // heading's own scroll margin, read from the stylesheet rather than assumed. + var landing = []; + function measureLanding() { + var pad = parseFloat(getComputedStyle(document.documentElement).scrollPaddingTop) || 0; + landing = tocTargets.map(function (t) { + return pad + (parseFloat(getComputedStyle(t.el).scrollMarginTop) || 0); + }); + } + + // The sidebar's contents: the h2 an h3 being read is in is marked too, and + // the sidebar scrolls (itself, never the page) to keep the mark in view. + var sideToc = document.querySelector(".docs-nav .page-toc"); + var sideMenu = sideToc ? sideToc.closest(".docs-menu") : null; + // Until the list has opened where it was (below), only the marks move. + var menuReady = false; + + // The current page's entry sticks at the top of the list over its own + // contents (site.css), so those are brought in below it. + var sideEntry = sideToc ? sideToc.parentElement.querySelector('a[aria-current="page"]') : null; + + function revealInMenu(el, whole) { + if (!sideMenu || !el || sideMenu.scrollHeight <= sideMenu.clientHeight + 1) return; + var m = sideMenu.getBoundingClientRect(); + if (!m.height) return; + var top = m.top; + if (sideEntry && sideToc.contains(el) && getComputedStyle(sideEntry).position === "sticky") top += sideEntry.offsetHeight; + var r = el.getBoundingClientRect(); + var pad = whole ? 8 : 40; + var by = 0; + if (r.height > m.bottom - top - 2 * pad || r.top < top + pad) by = r.top - top - pad; + else if (r.bottom > m.bottom - pad) by = r.bottom - m.bottom + pad; + if (Math.abs(by) < 1) return; + sideMenu.scrollTo({ top: sideMenu.scrollTop + by, behavior: reducedMotion() || whole ? "auto" : "smooth" }); + } + + // Near the foot of the page the end of the layout would push the sticky + // sidebar up and carry the mark out of sight; the list is cut short + // instead, so the sidebar stays whole and the mark stays in view. (150 is + // the room the stylesheet leaves around the list, calc(100vh - 150px).) + var sideNav = sideMenu ? sideMenu.closest(".docs-nav") : null; + var sideLayout = sideMenu ? sideMenu.closest(".docs-layout") : null; + var sideCap = ""; + function fitSide() { + if (!sideNav || !sideLayout) return; + var cap = ""; + if (getComputedStyle(sideNav).position === "sticky") { + var end = sideLayout.getBoundingClientRect().bottom - parseFloat(getComputedStyle(sideLayout).paddingBottom); + var natural = Math.min(sideMenu.scrollHeight, window.innerHeight - 150); + var bottom = parseFloat(getComputedStyle(sideNav).top) + sideNav.offsetHeight - sideMenu.offsetHeight + natural; + if (bottom > end) cap = Math.max(160, Math.floor(natural - (bottom - end))) + "px"; + } + if (cap === sideCap) return; + sideCap = cap; + sideMenu.style.maxHeight = cap; + var mark = sideToc.querySelector('a[aria-current="location"]'); + if (mark && menuReady) revealInMenu(mark, true); + } + + function markSide(link) { + if (!sideToc) return; + var group = link ? link.closest(".toc-2") : null; + sideToc.querySelectorAll(".toc-2").forEach(function (li) { + li.classList.toggle("in", li === group && !!link.closest(".toc-3")); + }); + if (link && menuReady) revealInMenu(link, false); + } + + // The main section a heading belongs to: the heading itself, or for a + // subsection the main section before it. + function sectionOf(id) { + var section = null; + for (var i = 0; i < tocTargets.length; i++) { + if (tocTargets[i].level === 2) section = tocTargets[i].id; + if (tocTargets[i].id === id) return section; + } + return null; + } + + // The addresses a contents link's own list holds (the sidebar's, the + // phone's box under the title, the sheet's, or the rail). + function listOf(link) { + if (!link._list) { + var root = link.closest(".page-toc, .toc-inline, .sheet-toc, .rail") || link.parentNode; + link._list = {}; + root.querySelectorAll("a").forEach(function (a) { + link._list[a.getAttribute("href")] = true; + }); + } + return link._list; + } + + function setCurrent(id) { + currentId = id; + // The sheet and the phone's box list the main sections only: while a + // subsection is read they mark the section it is in, so the bar that + // names it opens the contents at their place, not the keyboard. + var section = sectionOf(id); + var side = null; + tocLinks.forEach(function (link) { + var href = link.getAttribute("href"); + var on = href === "#" + id || (!!section && section !== id && href === "#" + section && !listOf(link)["#" + id]); + link.classList.toggle("active", on); + if (on) link.setAttribute("aria-current", "location"); + else link.removeAttribute("aria-current"); + if (on && sideToc && sideToc.contains(link)) side = link; + }); + markSide(side); + updateHere(); + } + + // The bar shows on a phone only; on a wider screen a scroll leaves it be. + var phoneWidth = window.matchMedia ? window.matchMedia("(max-width: 860px)") : null; + function updateHere() { + if (!hereBar || (phoneWidth && !phoneWidth.matches)) return; + var title = document.querySelector("main h1"); + var past = title ? title.getBoundingClientRect().bottom < HEADER_GAP : window.scrollY > 240; + hereBar.classList.toggle("on", past); + var current = tocTargets.filter(function (t) { + return t.id === currentId; + })[0]; + var text = current ? current.text : pageName(); + if (hereLabel.textContent !== text) { + hereLabel.textContent = text; + hereBar.setAttribute("aria-label", text + ": this page's contents, and search"); + } + var room = document.documentElement.scrollHeight - window.innerHeight; + var progress = room > 0 ? Math.min(1, Math.max(0, window.scrollY / room)) : 0; + hereProgress.style.transform = "scaleX(" + progress.toFixed(3) + ")"; + } + + // A click in the contents marks that link at once; the scroll it starts + // does not move the mark until it ends. + var locked = false; + var lockTimer = 0; + function lockMark() { + locked = true; + clearTimeout(lockTimer); + lockTimer = setTimeout(unlockMark, 700); + } + + function unlockMark() { + locked = false; + clearTimeout(lockTimer); + } + + if (tocTargets.length) { var ticking = false; var markCurrent = function () { ticking = false; - var current = 0; - targets.forEach(function (heading, k) { - if (heading.getBoundingClientRect().top < 120) current = k; - }); - tocLinks.forEach(function (link, k) { - link.classList.toggle("active", k === current); - }); + updateHere(); + fitSide(); + if (locked) return; + var doc = document.documentElement; + var scrolls = doc.scrollHeight > window.innerHeight + 2; + var atBottom = scrolls && window.innerHeight + window.scrollY >= doc.scrollHeight - 2; + var k = 0; + if (atBottom) { + k = tocTargets.length - 1; + } else { + tocTargets.forEach(function (t, j) { + if (t.el.getBoundingClientRect().top < landing[j] + 24) k = j; + }); + } + if (tocTargets[k].id !== currentId) setCurrent(tocTargets[k].id); }; + measureLanding(); window.addEventListener( "scroll", function () { + if (locked) { + // While a jump is under way, wait for it to settle. + clearTimeout(lockTimer); + lockTimer = setTimeout(unlockMark, 180); + } if (!ticking) { ticking = true; requestAnimationFrame(markCurrent); @@ -395,12 +2030,175 @@ }, { passive: true } ); - markCurrent(); + window.addEventListener("scrollend", function () { + if (locked) unlockMark(); + }); + window.addEventListener("resize", function () { + measureLanding(); + markCurrent(); + }); + tocLinks.forEach(function (link) { + link.addEventListener("click", function () { + var id = decodeURIComponent((link.getAttribute("href") || "").slice(1)); + if (!seenTarget[id]) return; + lockMark(); + setCurrent(id); + }); + }); + var arrivedAt = decodeURIComponent(location.hash.slice(1)); + if (arrivedAt && seenTarget[arrivedAt]) { + setCurrent(arrivedAt); + lockMark(); + } else { + markCurrent(); + } + } + + // โ”€โ”€ Arriving at a heading โ”€โ”€ + // The heading a link lands on carries a short accent bar, and the words of + // the search that brought the reader here are marked in its section. + + function targetOfHash() { + var id = decodeURIComponent(location.hash.slice(1)); + return id ? document.getElementById(id) : null; + } + + function arrive(target) { + if (!target || reducedMotion() || !/^H[1-6]$/.test(target.tagName)) return; + target.classList.remove("arrive"); + void target.offsetWidth; + target.classList.add("arrive"); + setTimeout(function () { + target.classList.remove("arrive"); + }, 1200); + } + + function headingLevel(node) { + if (!node || node.nodeType !== 1) return 0; + var h = /^H[1-6]$/.test(node.tagName) ? node : node.classList.contains("hx") ? node.querySelector("h1, h2, h3, h4, h5, h6") : null; + return h ? +h.tagName.charAt(1) : 0; } - // On a phone the page list starts folded, so the page itself comes first. - var docsMenu = document.querySelector(".docs-menu"); - if (docsMenu && window.matchMedia && window.matchMedia("(max-width: 860px)").matches) { - docsMenu.open = false; + function markWords(root, words, limit) { + var walker = document.createTreeWalker(root, NodeFilter.SHOW_TEXT, { + acceptNode: function (node) { + return node.parentElement.closest("pre, script, style, svg, button, .anchor, mark, kbd") ? NodeFilter.FILTER_REJECT : NodeFilter.FILTER_ACCEPT; + } + }); + var nodes = []; + while (walker.nextNode()) nodes.push(walker.currentNode); + var made = 0; + nodes.forEach(function (node) { + if (made >= limit) return; + var found = ranges(node.nodeValue, words); + if (!found.length) return; + var frag = document.createDocumentFragment(); + var at = 0; + found.forEach(function (r) { + if (made >= limit) return; + frag.appendChild(document.createTextNode(node.nodeValue.slice(at, r[0]))); + var mark = document.createElement("mark"); + mark.className = "search-hit"; + mark.textContent = node.nodeValue.slice(r[0], r[1]); + frag.appendChild(mark); + at = r[1]; + made++; + }); + frag.appendChild(document.createTextNode(node.nodeValue.slice(at))); + node.parentNode.replaceChild(frag, node); + }); + } + + function markSearchArrival() { + var hit = sessionGet("amap.searchHit"); + if (!hit || !hit.url || !hit.words || !hit.words.length) return; + if (hit.url.split("#")[0] !== location.href.split("#")[0] || (hit.url.split("#")[1] || "") !== location.hash.slice(1)) return; + sessionSet("amap.searchHit", null); + var main = document.querySelector("main"); + var start = targetOfHash() || (main && main.querySelector("h1")); + if (!start) return; + var level = headingLevel(start); + if (!level) { + markWords(start, hit.words, 40); + return; + } + var node = start.parentElement && start.parentElement.classList.contains("hx") ? start.parentElement : start; + var section = [node]; + for (var next = node.nextElementSibling; next; next = next.nextElementSibling) { + var l = headingLevel(next); + if (l && l <= Math.max(level, 2)) break; + section.push(next); + } + var left = 40; + section.forEach(function (el) { + markWords(el, hit.words, left); + left = 40 - document.querySelectorAll("mark.search-hit").length; + }); + } + + arrive(targetOfHash()); + markSearchArrival(); + window.addEventListener("hashchange", function () { + arrive(targetOfHash()); + markSearchArrival(); + }); + + // โ”€โ”€ The page list keeps the current page in view โ”€โ”€ + // The list scrolls on its own; it opens where it was on the page before, + // and if the current page is out of sight there, it is brought in with as + // much of its contents as fits, then the section being read. + var menu = document.querySelector(".docs-menu"); + if (menu) { + // On a phone the list starts folded, so the page itself comes first. A + // page that folds it before the first paint (a script right after the + // list) is left alone; any other page is folded here. + var foldsItself = menu.nextElementSibling && menu.nextElementSibling.tagName === "SCRIPT"; + if (!foldsItself && window.matchMedia && window.matchMedia("(max-width: 860px)").matches) menu.open = false; + var currentPage = menu.querySelector('a[aria-current="page"]'); + if (menu.scrollHeight > menu.clientHeight + 1) { + var saved = sessionGet("amap.menuScroll"); + if (typeof saved === "number") menu.scrollTop = saved; + if (currentPage) { + var m = menu.getBoundingClientRect(); + var r = currentPage.getBoundingClientRect(); + if (r.top < m.top + 8 || r.bottom > m.bottom - 8) { + menu.scrollTop += r.top - m.top - (menu.clientHeight - r.height) / 2; + } + if (sideMenu === menu) { + revealInMenu(currentPage.parentElement, true); + revealInMenu(sideToc.querySelector('a[aria-current="location"]'), true); + } + } + } + menuReady = true; + var saveMenu = function () { + sessionSet("amap.menuScroll", menu.scrollTop); + }; + menu.addEventListener("scroll", saveMenu, { passive: true }); + window.addEventListener("pagehide", saveMenu); + // While more of the list is below its edge, the edge fades, rather than + // cut a page's name in half as if the list ended there. + var moreBelow = function () { + menu.classList.toggle("more-below", menu.scrollTop + menu.clientHeight < menu.scrollHeight - 2); + }; + moreBelow(); + menu.addEventListener("scroll", moreBelow, { passive: true }); + window.addEventListener("resize", moreBelow); + } + + // โ”€โ”€ The header's "you are here" โ”€โ”€ + // On the front page, "Start here" is current while its section is on screen. + var startLink = document.querySelector('.top .nav a[href="#start"]'); + var startSection = document.getElementById("start"); + if (startLink && startSection && "IntersectionObserver" in window) { + new IntersectionObserver( + function (entries) { + entries.forEach(function (entry) { + if (entry.isIntersecting) startLink.setAttribute("aria-current", "location"); + else startLink.removeAttribute("aria-current"); + }); + }, + { rootMargin: "-45% 0px -45% 0px" } + ).observe(startSection); } })(); diff --git a/docs/assets/social-card.png b/docs/assets/social-card.png index 9641a60..9a4939a 100644 Binary files a/docs/assets/social-card.png and b/docs/assets/social-card.png differ diff --git a/docs/assets/touch-icon.png b/docs/assets/touch-icon.png index 0ecf6d5..170626f 100644 Binary files a/docs/assets/touch-icon.png and b/docs/assets/touch-icon.png differ diff --git a/docs/deployment/containers.html b/docs/deployment/containers.html index 6c09bcd..f4d0571 100644 --- a/docs/deployment/containers.html +++ b/docs/deployment/containers.html @@ -4,139 +4,160 @@ Containers | AMAP docs - + + - + - + - + - +
    - AMAP + AMAP
    - + + +
    -

    Docs / Deployment

    +

    Docs / Deployment

    Describes AMAP on main, ahead of 1.0.1: what changed

    Containers

    -

    Every tool AMAP calls lives in one image, paururo/amap, published on Docker Hub. Both the docker and singularity profiles use it, so nothing needs installing beyond a container engine.

    -
    nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
-nextflow run PathoGenOmics-Lab/AMAP -profile singularity --tsv samples.tsv
    -

    Nextflow pulls the image on first use and caches it. Singularity converts it to a .sif on the way, which takes a few minutes once.

    -

    What is inside#

    -

    What you install

    • Nextflow 24.04 or newer, on Java 17 or newer
    • Docker, or Singularity or Apptainer on a cluster

    What the first run builds

    • The eggNOG database, in the folder --eggnog_db names, kept for every run after

    paururo/amaplinux/amd64

    Tools

    Bakta 1.12.0miniprot 0.18Diamond 2.1.24eggNOG-mapper 2.1.13pangene 1.1pathotypr 1.0.0gfatools

    Runtimes

    Python 3.9R 4.4

    Data

    Bakta database light/reference/ 9 files: the H37Rv genome, proteome and GenBank, Bakta's false positives on it, Mycobrowser's table and the lineage markers
    Two things to install; everything AMAP runs comes in one image, its versions read off docker/Dockerfile. Only the eggNOG database stays outside: too large to ship, it is built on the first run that needs it.
    -

    Tools: Bakta 1.12.0, Miniprot 0.18, Diamond 2.1.24, eggNOG-mapper 2.1.13, pangene 1.1, pathotypr 1.0.0, gfatools.

    +

    The AMAP image, what is in it, and building your own.

    +

    3 min read

    +
    On this page
    + +

    Every tool AMAP calls lives in one image, paururo/amap, published on Docker Hub. Both the docker and singularity profiles use it, so nothing needs installing beyond a container engine.

    +
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
+$ nextflow run PathoGenOmics-Lab/AMAP -profile singularity --tsv samples.tsv
    +

    Nextflow pulls the image on first use and caches it. Singularity converts it to an image file of its own on the way, which takes a few minutes once per cache folder: see Singularity on a cluster for keeping one.

    +

    What is inside

    #
    +

    What you install

    • Nextflow 24.04 or newer, on Javaย 17 or newer
    • Docker, or Singularity or Apptainer on a cluster

    What the first run builds

    • The eggNOG database, in the folder --eggnog_db names, kept for every run after

    paururo/amaplinux/amd64

    Tools

    Bakta 1.12.0miniprot 0.18Diamond 2.1.24eggNOG-mapper 2.1.13pangene 1.1pathotypr 1.0.0gfatools

    Runtimes

    Python 3.9R 4.4

    Data

    Bakta database light/reference/ 9 files: the H37Rv genome, proteome and GenBank, Bakta's false positives on it, Mycobrowser's table and the lineage markers
    Two things to install; everything AMAP runs comes in one image, its versions read off docker/Dockerfile. Only the eggNOG database stays outside: too large to ship, it is built on the first run that needs it.
    +

    Tools: Bakta 1.12.0, miniprot 0.18, Diamond 2.1.24, eggNOG-mapper 2.1.13, pangene 1.1, pathotypr 1.0.0, gfatools.

    Runtimes: Python 3.9 with pandas, NumPy, Biopython, matplotlib, plotly, NetworkX, intervaltree, pyvis and python-louvain. R 4.4 with the report packages.

    -

    Bundled data under /reference/, so a standard MTBC run needs no downloads:

    -
    FileWhat it is
    MTB_ancestor_reference.fastaMTBC ancestral reference genome
    aa_H37Rv.translated.fasH37Rv reference proteome
    bad.bakta.translated.fasCurated Bakta false positives
    new_refH37Rv.fastaH37Rv reference genome
    sequence.gbH37Rv GenBank, for Bakta
    Mycobacterium_tuberculosis_H37Rv_txt_v4.txtMycobrowser functional categories
    snps_lineages.tsvpathotypr lineage markers
    -

    Plus the Bakta light database at /data/bakta_db.

    -

    The eggNOG database is not bundled, because of its size. AMAP downloads and builds it on first use and caches it at --eggnog_db.

    -

    Platform#

    -

    The image is linux/amd64 only, and every Dockerfile stage pins it. On Apple Silicon it runs under emulation, which works but is slow. For anything past a couple of genomes, use an amd64 machine or a cluster.

    -

    Pinning a version#

    -

    latest tracks the newest release. For a reproducible analysis, pin the version in nextflow.config or on the command line:

    -
    nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv \
-    -process.container 'paururo/amap:1.0.1'
    -

    results/pipeline_info/software_versions.yml records the exact version of every tool a run used, which is what belongs in a methods section.

    -

    Building it yourself#

    +

    Bundled data under /reference/, so a standard run on the Mycobacterium tuberculosis complex (MTBC) needs no downloads:

    +
    FileWhat it is
    MTB_ancestor_reference.fastaMTBC ancestral reference genome
    aa_H37Rv.translated.fasH37Rv reference proteome
    bad.bakta.translated.fasCurated Bakta false positives
    new_refH37Rv.fastaH37Rv reference genome
    sequence.gbH37Rv GenBank, for Bakta
    Mycobacterium_tuberculosis_H37Rv_txt_v4.txtMycobrowser functional categories
    snps_lineages.tsvpathotypr lineage markers
    +

    Plus the Bakta light database at /data/bakta_db.

    +

    The eggNOG database is not bundled, because of its size. AMAP downloads and builds it on first use and caches it at --eggnog_db.

    +

    Platform

    #
    +

    The image is linux/amd64 only, and every Dockerfile stage pins it. On Apple Silicon it runs under emulation, which works but is slow. For anything past a couple of genomes, use an amd64 machine or a cluster.

    +

    Pinning aย version

    #
    +

    A tag can move: latest is whatever was pushed last. For a reproducible analysis, pin the image by its digest, and the pipeline's code with -r, since a pinned image under newer code is still a different run:

    +
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP \
+    -r 947a99dee49d2a2429e014296ac35ba08e911108 \
+    -profile docker --tsv samples.tsv \
+    -process.container 'paururo/amap@sha256:08bd17311ab323317ceecba6e5bc639a29f4c54c9d95f9a7c48ba80a9fc640ee'
    +

    That digest is the image behind the runs on these pages; docker image inspect paururo/amap:latest --format '{{index .RepoDigests 0}}' prints the one you have. Pinning the pipeline has the rest.

    +

    results/pipeline_info/software_versions.yml records the exact version of every tool a run used, which is what belongs in a methods section.

    +

    Building it yourself

    #

    Only needed if you are changing what is in the image.

    -
    git clone https://github.com/PathoGenOmics-Lab/AMAP
-cd AMAP
-docker build --platform linux/amd64 -f docker/Dockerfile -t amap:dev .
    +
    Terminal
    $ git clone https://github.com/PathoGenOmics-Lab/AMAP
+$ cd AMAP
+$ docker build --platform linux/amd64 -f docker/Dockerfile -t amap:dev .

    The build takes about an hour: it downloads the Bakta database and solves a conda environment with R. Check the result before using it:

    -
    docker/smoke_test.sh amap:dev
    +
    Terminal
    $ docker/smoke_test.sh amap:dev

    That verifies the tools, the Python and R libraries, the bundled reference data, and runs the repository test suite with the image interpreter. The last part matters, since a library update can break the pipeline scripts without the image looking any different.

    -

    Publishing is automated: the Docker image GitHub Actions workflow builds, smoke tests and pushes on a v* tag or on demand.

    -

    Reference downloads can expire

    Several reference files are fetched from Google Drive during the build. Those links have expired before. If a build fails on a wget in the data stage, that is why, and the URL needs replacing. The Zenodo-hosted files (MTB_ancestor_reference.fasta, the pathotypr markers) are not affected.

    -

    Singularity on a cluster#

    -

    Set a shared cache so every job reuses one converted image instead of each building its own:

    -
    export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache
    -

    Put it in your shell profile or the launch script. Without it, a large run can have dozens of jobs converting the same image at once.

    -

    More cluster specifics in deployment on HPC.

    -

    Next steps#

    - -

    Edit this page on GitHub

    - +

    Publishing is automated: the Docker image GitHub Actions workflow builds, smoke tests and pushes on a v* tag or on demand.

    +

    WarningReference downloads can expire

    Several reference files are fetched from Google Drive during the build. Those links have expired before. If a build fails on a wget in the data stage, that is why, and the URL needs replacing. The Zenodo-hosted files (MTB_ancestor_reference.fasta, the pathotypr markers) are not affected.

    +

    Singularity on aย cluster

    #
    +

    The singularity profile keeps the converted image in <outdir>/singularity_cache, so each results folder gets a copy of its own, and that setting wins over NXF_SINGULARITY_CACHEDIR: the variable alone changes nothing. To keep one copy for every run, set the folder in a config and pass it with -c:

    +
    shared.config
    singularity.cacheDir = '/shared/singularity_cache'
    +
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
+    -c shared.config --tsv samples.tsv
    +

    With Nextflowย 24.04 and 26.10 alike, nextflow -c shared.config config PathoGenOmics-Lab/AMAP -profile singularity then shows cacheDir = '/shared/singularity_cache', where without it the profile's results/singularity_cache stays, whatever the variable says. The pipeline has to be pulled first: any run does it, or nextflow pull PathoGenOmics-Lab/AMAP.

    +

    More cluster specifics in HPC clusters.

    +

    Next steps

    #
    + +

    Back to the top

    +

    Edit this page on GitHub

    +
    - - diff --git a/docs/deployment/containers.md b/docs/deployment/containers.md index b5e9709..c391381 100644 --- a/docs/deployment/containers.md +++ b/docs/deployment/containers.md @@ -10,21 +10,23 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv nextflow run PathoGenOmics-Lab/AMAP -profile singularity --tsv samples.tsv ``` -Nextflow pulls the image on first use and caches it. Singularity converts it to -a `.sif` on the way, which takes a few minutes once. +Nextflow pulls the image on first use and caches it. Singularity converts it +to an image file of its own on the way, which takes a few minutes once per +cache folder: see [Singularity on a cluster](#singularity-on-a-cluster) for +keeping one. ## What is inside -**Tools**: Bakta 1.12.0, Miniprot 0.18, Diamond 2.1.24, eggNOG-mapper 2.1.13, +**Tools**: Bakta 1.12.0, miniprot 0.18, Diamond 2.1.24, eggNOG-mapper 2.1.13, pangene 1.1, pathotypr 1.0.0, gfatools. **Runtimes**: Python 3.9 with pandas, NumPy, Biopython, matplotlib, plotly, NetworkX, intervaltree, pyvis and python-louvain. R 4.4 with the report packages. -**Bundled data** under `/reference/`, so a standard MTBC run needs no downloads: +**Bundled data** under `/reference/`, so a standard run on the *Mycobacterium tuberculosis* complex (MTBC) needs no downloads: | File | What it is | | :--- | :--- | @@ -49,15 +51,22 @@ couple of genomes, use an amd64 machine or a cluster. ## Pinning a version -`latest` tracks the newest release. For a reproducible analysis, pin the -version in `nextflow.config` or on the command line: +A tag can move: `latest` is whatever was pushed last. For a reproducible +analysis, pin the image by its digest, and the pipeline's code with `-r`, since +a pinned image under newer code is still a different run: ```bash -nextflow run PathoGenOmics-Lab/AMAP -profile docker \ - --tsv samples.tsv \ - -process.container 'paururo/amap:1.0.1' +nextflow run PathoGenOmics-Lab/AMAP \ + -r 947a99dee49d2a2429e014296ac35ba08e911108 \ + -profile docker --tsv samples.tsv \ + -process.container 'paururo/amap@sha256:08bd17311ab323317ceecba6e5bc639a29f4c54c9d95f9a7c48ba80a9fc640ee' ``` +That digest is the image behind the runs on these pages; +`docker image inspect paururo/amap:latest --format '{{index .RepoDigests 0}}'` +prints the one you have. [Pinning the pipeline](../user-guide/running.md#pinning-the-pipeline) +has the rest. + `results/pipeline_info/software_versions.yml` records the exact version of every tool a run used, which is what belongs in a methods section. @@ -96,19 +105,30 @@ smoke tests and pushes on a `v*` tag or on demand. ## Singularity on a cluster -Set a shared cache so every job reuses one converted image instead of each -building its own: +The `singularity` profile keeps the converted image in +`/singularity_cache`, so each results folder gets a copy of its own, +and that setting wins over `NXF_SINGULARITY_CACHEDIR`: the variable alone +changes nothing. To keep one copy for every run, set the folder in a config +and pass it with `-c`: + +```groovy title="shared.config" +singularity.cacheDir = '/shared/singularity_cache' +``` ```bash -export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache +nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \ + -c shared.config --tsv samples.tsv ``` -Put it in your shell profile or the launch script. Without it, a large run can -have dozens of jobs converting the same image at once. +With Nextflow 24.04 and 26.10 alike, `nextflow -c shared.config config +PathoGenOmics-Lab/AMAP -profile singularity` then shows +`cacheDir = '/shared/singularity_cache'`, where without it the profile's +`results/singularity_cache` stays, whatever the variable says. The pipeline has +to be pulled first: any run does it, or `nextflow pull PathoGenOmics-Lab/AMAP`. -More cluster specifics in [deployment on HPC](hpc.md). +More cluster specifics in [HPC clusters](hpc.md). ## Next steps -- [HPC deployment](hpc.md): scratch, shared filesystems, schedulers +- [HPC clusters](hpc.md): scratch, shared filesystems, schedulers - [Troubleshooting](troubleshooting.md): when a run fails diff --git a/docs/deployment/hpc.html b/docs/deployment/hpc.html index 70740be..cef95ac 100644 --- a/docs/deployment/hpc.html +++ b/docs/deployment/hpc.html @@ -4,169 +4,195 @@ HPC clusters | AMAP docs - + + - + - + - + - +
    - AMAP + AMAP
    - + + +
    -

    Docs / Deployment

    -

    Running on a cluster

    -

    Past a couple of dozen genomes, a scheduler is worth the setup. Combine a container profile with an executor profile:

    -
    nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv
    -

    The launcher

    nextflow run, in tmux or as a small job of its own: 2 CPUs, 8 GB, 48 h. It submits, waits and resubmits.

    Runs here too: VALIDATE_INPUTS, SOFTWARE_VERSIONS, DUMP_PARAMS

    The scheduler

    Up to 200 jobs at once, submitted at up to 100 a minute with slurm (20 a minute with sge, pbs and lsf).

    The compute nodes

    One job per task, at most 48 CPUs and 128 GB each, in the AMAP image, working on the node's own disk and copying back only the results.

    per sample: Bakta, miniprot, the correction, duplications, pangenomeonce for all: eggNOG, the genomes compared, the report

    Shared storage

    -work-direvery task's files, which -resume needsNXF_SINGULARITY_CACHEDIRthe image, converted once--eggnog_dbthe eggNOG database, built once--outdirthe results
    A run on a cluster, with the settings of the slurm profile in nextflow.config and the launcher of the job script below.
    -

    The executor profiles#

    -
    ProfileSchedulerSubmission rate
    slurmSLURM100 per minute
    sgeSun Grid Engine, Univa20 per minute
    pbsPBS, Torque20 per minute
    lsfIBM Spectrum LSF20 per minute
    -

    All four queue up to 200 jobs at once, default to 48 CPUs and 128 GB per job, and enable node-local scratch. SGE additionally requests memory through -l h_vmem, and uses the smp parallel environment.

    -

    Tune the concurrency to your site's policy if 200 is too aggressive:

    -
    nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
-    --tsv samples.tsv -process.queueSize 50
    -

    Fitting your site#

    -
    nextflow run PathoGenOmics-Lab/AMAP \
-    -profile singularity,slurm \
-    --tsv samples.tsv \
-    --max_cpus 64 --max_memory 256.GB \
-    -process.queue long \
-    -process.clusterOptions '--account=mygroup'
    -

    -process.queue sets the partition or queue, and -process.clusterOptions passes anything else straight through, which is where accounting flags, QoS and node constraints go.

    -

    On SLURM, Bakta's jobs, the longest a run sends, also ask for --qos=medium. The slurm profile sets that for Bakta alone, and for Bakta it takes the place of -process.clusterOptions, so give Bakta your site's options in a config passed with -c:

    -
    process {
+
    Docs / Deployment
    Describes AMAP on main, ahead of 1.0.1: what changed
    
+
    HPC clusters
    
+
    Schedulers, scratch, and where things should live.
    
+
    3 min read
    
+
    On this page
    
+
+
    Past about twenty genomes, a scheduler is worth the setup. Combine a container profile with an executor profile:
    
+
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv
    
+
    The launcher
    nextflow run, in tmux or as a small job of its own: 2ย CPUs, 8ย GB, 48ย h. It submits, waits and resubmits.
    Runs here too: VALIDATE_INPUTS, SOFTWARE_VERSIONS, DUMP_PARAMS
    The scheduler
    Up to 200 jobs at once, submitted at up to 100 a minute with slurm (20 a minute with sge, pbs and lsf).
    The compute nodes
    One job per task, at most 48ย CPUs and 128ย GB each, in the AMAP image, working on the node's own disk and copying back only the results.
    per sample: Bakta, miniprot, the correction, duplicationsonce for all: eggNOG, the pangenome, the genomes compared, the report
    Shared storage
    -work-direvery task's files, which -resume needssingularity.cacheDirthe image, converted once, from a -c config--eggnog_dbthe eggNOG database, built once--outdirthe results
    A run on a cluster, with the settings of the slurm profile in nextflow.config and the launcher of the job script below.
    
+
    The executor profiles
    #
    
+
    ProfileSchedulerSubmission rate
    slurmSLURM100 per minute
    sgeSun Grid Engine, Univa20 per minute
    pbsPBS, Torque20 per minute
    lsfIBM Spectrum LSF20 per minute
    
+
    All four queue up to 200 jobs at once, default to 48ย CPUs and 128ย GB per job, and enable node-local scratch. SGE additionally requests memory through -l h_vmem, and uses the smp parallel environment.
    
+
    Tune the concurrency to your site's policy if 200 is too aggressive. The queue size belongs to the executor, not to a process, so -process.queueSize changes nothing; set it in a config passed with -c:
    
+
    queue.config
    executor.queueSize = 50
    
+
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
+    -c queue.config --tsv samples.tsv
    
+
    Fitting your site
    #
    
+
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP \
+    -profile singularity,slurm \
+    --tsv samples.tsv \
+    --max_cpus 64 --max_memory 256.GB \
+    -process.queue long \
+    -process.clusterOptions '--account=mygroup'
    
+
    -process.queue sets the partition or queue, and -process.clusterOptions passes anything else straight through, which is where accounting flags, QoS and node constraints go.
    
+
    On SLURM, Bakta's jobs, the longest a run sends, also ask for --qos=medium. The slurm profile sets that for Bakta alone, and for Bakta it takes the place of -process.clusterOptions, so give Bakta your site's options in a config of your own, with --qos=medium if your cluster has that QoS and without it if not:
    
+
    bakta.config
    process {
     withName: 'BAKTA_ANNOTATOR' {
         clusterOptions = '--qos=medium --account=mygroup'
     }
 }
    
-
    Scratch#
    
-
    --scratch is on by default in every HPC profile, and it matters. Without it, every process reads and writes directly on the shared filesystem, and Bakta and eggNOG in particular generate enough metadata traffic to slow a cluster down for everyone. With it, work happens on node-local disk and only the results are copied back.
    
+
    Then pass it with -c, next to the options for every other job:
    
+
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
+    -c bakta.config --tsv samples.tsv \
+    -process.clusterOptions '--account=mygroup'
    
+
    Scratch
    #
    
+
    --scratch is on by default in every executor profile, and it matters. Without it, every process reads and writes directly on the shared filesystem, and Bakta and eggNOG in particular generate enough metadata traffic to slow a cluster down for everyone. With it, work happens on node-local disk and only the results are copied back.
    
 
    Turn it off only if your nodes have no local disk:
    
-
    nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
-    --tsv samples.tsv --scratch false
    
-
    Where to put things#
    
-
    The Singularity cache. Set it once, somewhere shared:
    
-
    export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache
    
-
    Otherwise the singularity profile caches under --outdir, and a large run can have many jobs converting the same image at once.
    
+
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
+    --tsv samples.tsv --scratch false
    
+
    Where to put things
    #
    
+
    The Singularity cache. The singularity profile keeps the converted image in <outdir>/singularity_cache, so every results folder gets a copy of its own, and that setting wins over NXF_SINGULARITY_CACHEDIR: the variable alone changes nothing. Keep one shared copy with a config passed with -c:
    
+
    shared.config
    singularity.cacheDir = '/shared/singularity_cache'
    
+
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
+    -c shared.config --tsv samples.tsv
    
 
    The eggNOG database. Large, built once, reused forever. Point it at persistent shared storage rather than somewhere per-run:
    
-
    nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
-    --tsv samples.tsv --eggnog_db /shared/databases/eggnog
    
+
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
+    --tsv samples.tsv --eggnog_db /shared/databases/eggnog
    
 
    The work directory. It gets large, tens of gigabytes for a big run. Put it on a filesystem with room and a generous quota:
    
-
    nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
-    --tsv samples.tsv -work-dir /scratch/$USER/amap_work
    
-
    Keep it until you are sure the run is finished, since -resume needs it.
    
-
    Launching#
    
-
    Nextflow's own process must survive the whole run, which will outlive an SSH session. Either run it under tmux or screen, or submit it as a small job of its own:
    
-
    run_amap.sh
    #!/bin/bash
-#SBATCH --job-name=amap
-#SBATCH --cpus-per-task=2
-#SBATCH --mem=8G
-#SBATCH --time=48:00:00
-#SBATCH --output=amap_%j.log
-
-module load singularity
-export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache
-
-nextflow run PathoGenOmics-Lab/AMAP \
-    -profile singularity,slurm \
-    --tsv samples.tsv \
-    --eggnog_db /shared/databases/eggnog \
-    -work-dir /scratch/$USER/amap_work \
-    -resume
    
+
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
+    --tsv samples.tsv -work-dir /scratch/$USER/amap_work
    
+
    Keep it until you are sure the run is finished, since -resume needs it.
    
+
    Launching
    #
    
+
    Nextflow's own process must survive the whole run, which will outlive an SSH session. Either run it under tmux or screen, or submit it as a small job of its own:
    
+
    run_amap.sh
    #!/bin/bash
+#SBATCH --job-name=amap
+#SBATCH --cpus-per-task=2
+#SBATCH --mem=8G
+#SBATCH --time=48:00:00
+#SBATCH --output=amap_%j.log
+
+module load singularity
+
+nextflow run PathoGenOmics-Lab/AMAP \
+    -profile singularity,slurm \
+    -c shared.config \
+    --tsv samples.tsv \
+    --eggnog_db /shared/databases/eggnog \
+    -work-dir /scratch/$USER/amap_work \
+    -resume
    
 
    The launcher itself needs very little: it submits and waits while the real work happens in the jobs it spawns. Give it plenty of wall time, though, since it has to outlast everything it submits.
    
-
    Memory and retries#
    
-
    A process killed by the out-of-memory killer is resubmitted with more memory, scaling with the attempt number up to --max_memory. That handles the occasional unusually large genome without you having to size everything for the worst case.
    
-
    eggNOG is the memory-hungry step. It runs once over all samples rather than once per sample, which keeps a shared filesystem from being hammered but does mean one large job. The highmem profile exists for this:
    
-
    nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm,highmem \
-    --tsv samples.tsv
    
-
    Watching a run#
    
-
    squeue -u $USER shows the jobs. When it finishes, pipeline_info/ holds the execution report, the timeline, and a trace with per-process CPU, memory and runtime, which is what to look at before adjusting resources for the next run.
    
-
    Next steps#
    
+
    Memory and retries
    #
    
+
    A process killed by the out-of-memory killer is resubmitted with more memory, scaling with the attempt number up to --max_memory. That handles the occasional unusually large genome without you having to size everything for the worst case.
    
+
    A task run per sample that fails for another reason is tried twice more and then set aside, so the run finishes without that sample and says so under Ignored: see a sample is missing.
    
+
    eggNOG is the memory-hungry step. It runs once over all samples rather than once per sample, which keeps a shared filesystem from being hammered but does mean one large job. The highmem profile exists for this:
    
+
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm,highmem \
+    --tsv samples.tsv
    
+
    Watching aย run
    #
    
+
    squeue -u $USER shows the jobs. When it finishes, pipeline_info/ holds the execution report, the timeline, and a trace with per-process CPU, memory and runtime, which is what to look at before adjusting resources for the next run.
    
+
    Next steps
    #
    
 
-
    Edit this page on GitHub
    
-
+
    Back to the top
    
+
    Edit this page on GitHub
    
+
    - - diff --git a/docs/deployment/hpc.md b/docs/deployment/hpc.md index 7a0f25d..ab45483 100644 --- a/docs/deployment/hpc.md +++ b/docs/deployment/hpc.md @@ -1,7 +1,7 @@ -# Running on a cluster +# HPC clusters -Past a couple of dozen genomes, a scheduler is worth the setup. Combine a -container profile with an executor profile: +Past about twenty genomes, a scheduler is worth the setup. Combine a container +profile with an executor profile: ```bash nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv @@ -22,11 +22,17 @@ All four queue up to 200 jobs at once, default to 48 CPUs and 128 GB per job, and enable node-local scratch. SGE additionally requests memory through `-l h_vmem`, and uses the `smp` parallel environment. -Tune the concurrency to your site's policy if 200 is too aggressive: +Tune the concurrency to your site's policy if 200 is too aggressive. The +queue size belongs to the executor, not to a process, so `-process.queueSize` +changes nothing; set it in a config passed with `-c`: + +```groovy title="queue.config" +executor.queueSize = 50 +``` ```bash nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \ - --tsv samples.tsv -process.queueSize 50 + -c queue.config --tsv samples.tsv ``` ## Fitting your site @@ -46,10 +52,11 @@ node constraints go. On SLURM, Bakta's jobs, the longest a run sends, also ask for `--qos=medium`. The `slurm` profile sets that for Bakta alone, and for Bakta it takes the place -of `-process.clusterOptions`, so give Bakta your site's options in a config -passed with `-c`: +of `-process.clusterOptions`, so give Bakta your site's options in a config of +your own, with `--qos=medium` if your cluster has that QoS and without it if +not: -```groovy +```groovy title="bakta.config" process { withName: 'BAKTA_ANNOTATOR' { clusterOptions = '--qos=medium --account=mygroup' @@ -57,9 +64,17 @@ process { } ``` +Then pass it with `-c`, next to the options for every other job: + +```bash +nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \ + -c bakta.config --tsv samples.tsv \ + -process.clusterOptions '--account=mygroup' +``` + ## Scratch -`--scratch` is on by default in every HPC profile, and it matters. Without it, +`--scratch` is on by default in every executor profile, and it matters. Without it, every process reads and writes directly on the shared filesystem, and Bakta and eggNOG in particular generate enough metadata traffic to slow a cluster down for everyone. With it, work happens on node-local disk and only the results are @@ -74,14 +89,19 @@ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \ ## Where to put things -**The Singularity cache.** Set it once, somewhere shared: +**The Singularity cache.** The `singularity` profile keeps the converted image +in `/singularity_cache`, so every results folder gets a copy of its +own, and that setting wins over `NXF_SINGULARITY_CACHEDIR`: the variable alone +changes nothing. Keep one shared copy with a config passed with `-c`: -```bash -export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache +```groovy title="shared.config" +singularity.cacheDir = '/shared/singularity_cache' ``` -Otherwise the `singularity` profile caches under `--outdir`, and a large run can -have many jobs converting the same image at once. +```bash +nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \ + -c shared.config --tsv samples.tsv +``` **The eggNOG database.** Large, built once, reused forever. Point it at persistent shared storage rather than somewhere per-run: @@ -116,10 +136,10 @@ its own: #SBATCH --output=amap_%j.log module load singularity -export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache nextflow run PathoGenOmics-Lab/AMAP \ -profile singularity,slurm \ + -c shared.config \ --tsv samples.tsv \ --eggnog_db /shared/databases/eggnog \ -work-dir /scratch/$USER/amap_work \ @@ -136,6 +156,10 @@ A process killed by the out-of-memory killer is resubmitted with more memory, scaling with the attempt number up to `--max_memory`. That handles the occasional unusually large genome without you having to size everything for the worst case. +A task run per sample that fails for another reason is tried twice more and +then set aside, so the run finishes without that sample and says so under +`Ignored`: see [a sample is missing](troubleshooting.md#a-sample-is-missing-from-the-results-ignored--1). + eggNOG is the memory-hungry step. It runs once over all samples rather than once per sample, which keeps a shared filesystem from being hammered but does mean one large job. The `highmem` profile exists for this: diff --git a/docs/deployment/troubleshooting.html b/docs/deployment/troubleshooting.html index 7c2b30f..89fe34b 100644 --- a/docs/deployment/troubleshooting.html +++ b/docs/deployment/troubleshooting.html @@ -4,148 +4,202 @@ Troubleshooting | AMAP docs - + + - + - + - + - +
    - AMAP + AMAP
    - + + +
    -

    Docs / Deployment

    +

    Docs / Deployment

    Describes AMAP on main, ahead of 1.0.1: what changed

    Troubleshooting

    -

    Reading a failure#

    +

    Organised by the error you are looking at.

    +

    6 min read

    +
    On this page
    + +

    Each problem below is a heading, with the message the pipeline or the system prints in quotes where there is one, so you can search for what you see.

    +

    Reading aย failure

    #

    When a process fails, Nextflow prints the work directory for the failed task. That directory has everything you need:

    -
    work/ab/cd1234.../
-โ”œโ”€โ”€ .command.sh      # the exact script that ran
-โ”œโ”€โ”€ .command.log     # everything it printed
-โ”œโ”€โ”€ .command.err     # stderr on its own
-โ””โ”€โ”€ .exitcode        # what it exited with
    -

    .command.err is almost always where the answer is. Re-running .command.sh inside the container reproduces the failure by hand.

    -

    results/pipeline_info/execution_trace_*.txt shows every task with its status, CPU, memory and runtime, which is the fastest way to see whether a failure was resources or logic.

    -

    Killed by the machine or the scheduler: exit 137, 140 or 143

    1. Tried again, up to 2 more times, asking for more memory each time: Bakta 6 GB, then 12 GB, then 18 GB.
    2. Never past --max_memory: a request above it is lowered to it.
    3. Still failing after that: the run stops. Raise --max_memory, or the memory Docker Desktop has.

    Any other exit, in a step run per sample

    1. Tried again, 2 more times.
    2. Still failing: that sample's task is set aside, and the run carries on with the others.
    3. Its work directory, named in the log, says why.

    The steps run once for all the samples are retried too, and then stop the run. Some stop it at once: VALIDATE_INPUTS on any failure, and the eggNOG annotation on anything but memory or time.

    What nextflow.config does with a task that fails.
    -

    Before the pipeline starts#

    -
    • "Unexpected input: '('" in nextflow.config: Nextflow 25.10 and later read pipelines with a stricter parser, which older copies of AMAP do not pass. Nextflow keeps the copy it downloaded first, so fetch the current one with nextflow pull PathoGenOmics-Lab/AMAP and run again. To run an older AMAP release as it is, export NXF_SYNTAX_PARSER=v1 first.
    • "Input TSV file not found": The path in --tsv is wrong, or relative to somewhere other than where you launched Nextflow.
    • "Invalid TSV format": The header must be exactly sampleId and fasta, separated by a real tab. Spaces are the usual culprit. cat -t samples.tsv shows tabs as ^I, and works on both Linux and macOS.
    • "FASTA file not found": A path in the sample sheet does not resolve. Absolute paths avoid the ambiguity.
    • "not found, neither on the host nor inside the container": A reference file the run reads is in neither place: a typo in its path, or one of the defaults, which ship in the container, on a run without one. Give the path to your own copy with its reference argument.
    • Validation failures: results/pipeline_info/validation.log says which check failed and for which sample. Thresholds are in preparing inputs.
    -

    Container problems#

    -

    The image will not pull. Check network access from the compute nodes, which often differs from the login node. Pre-pulling on the login node and setting NXF_SINGULARITY_CACHEDIR sidesteps it:

    -
    export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache
-singularity pull docker://paururo/amap:latest
    -

    Slow on Apple Silicon. The image is amd64 only, so it emulates. Fine for the test profile, painful beyond that. Use an amd64 machine.

    -

    Docker permission denied. Your user is not in the docker group. Either add it, or use Singularity.

    -

    Memory#

    -

    A process is killed with exit 137 or 140. Out of memory. AMAP retries with more, scaling by attempt, so a single occurrence in the log that later succeeds is normal. Persistent failures mean raising the ceiling:

    -
    --max_memory 256.GB
    -

    On a cluster, that is the whole answer. On a macOS or Windows laptop it is the wrong one: Docker Desktop runs containers inside a VM with its own memory ceiling, commonly 8 GB, and that ceiling is what killed the process. Nextflow sized the request against the host and never saw it. Raising --max_memory only makes the request larger than a limit it was already over. Raise the VM in Docker Desktop under Settings, Resources, Memory, then keep --max_memory at or below it.

    -

    eggNOG runs out of memory. It is the hungriest step, requesting 50 GB, and it runs once over all samples together. On a machine that cannot reach that, --skip_functional_annotation drops it along with the pangenome and the duplication analysis and still gives you the corrected annotation. On a cluster, use the highmem profile, or point it at a large-memory partition:

    -
    nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm,highmem \
-    --tsv samples.tsv
    -

    Filesystem#

    -

    "No space left on device". Usually the work directory, which grows to tens of gigabytes. Move it somewhere with room:

    -
    -work-dir /scratch/$USER/amap_work
    -

    Also check your $TMPDIR quota, since Diamond and Bakta both use it heavily.

    -

    Everything is slow on a shared filesystem. Confirm --scratch is on. It defaults to true in the HPC profiles, but a standard or docker profile on a cluster will not have it.

    -

    Cache misses on resume. NFS timestamp drift across nodes invalidates Nextflow's cache. The slurm profile already sets process.cache = 'lenient' for this. If you are using a different executor, add it:

    -
    -process.cache lenient
    -

    The eggNOG database#

    -

    It downloads every run. --eggnog_db is pointing somewhere per-run. Give it persistent shared storage.

    -

    It rebuilds itself unexpectedly. A bacteria.dmnd under 100 MB is treated as a truncated download and rebuilt. Usually means an earlier download was interrupted.

    -

    "has eggnog_db.ok, so EGGNOG_DB will not run again". eggnog_db.ok tells Nextflow the database is built, so it never runs EGGNOG_DB again, but what it needs is not all there. Delete the files the message names and the next run builds it again.

    -

    Results that look wrong#

    -

    Far fewer genes than expected. Check the assembly. A fragmented assembly loses genes across contig boundaries, and they turn up in *-rvmissing.tsv rather than being corrected. *-AMAP.qc.tsv has the contig count and N50.

    -

    "Genes lost to (Start, End, Strand, Contig) collisions" in the log. Two of the classification's sources listed a gene at exactly the same place, and one entry was kept. Genes on different contigs never collide, since the contig is part of the key.

    -

    *-AMAP.clean.faa has fewer sequences than the GFF3 has genes. Expected. The clean proteome drops pseudogenes and internal-stop sequences. Use *-AMAP.faa if you need one protein per annotated gene.

    -

    Almost everything is case 1. Your samples are too far from the reference for the alignments to find anything, so there is nothing to correct against. Use a closer reference. See custom reference.

    -

    The Mycobrowser sections are empty. Expected with --skip_mycobrowser, or when the Mycobrowser file is missing. The pipeline warns and continues.

    -

    Still stuck#

    +
    work/ab/cd1234.../
+โ”œโ”€โ”€ .command.sh      # the exact script that ran
+โ”œโ”€โ”€ .command.log     # everything it printed
+โ”œโ”€โ”€ .command.err     # stderr on its own
+โ””โ”€โ”€ .exitcode        # what it exited with
    +

    .command.err is almost always where the answer is. Re-running .command.sh inside the container reproduces the failure by hand.

    +

    results/pipeline_info/execution_trace_*.txt shows every task with its status, CPU, memory and runtime, which is the fastest way to see whether a failure was resources or logic.

    +

    Killed by the machine or the scheduler: exit 137, 140 or 143

    1. Tried again, up to 2 more times, asking for more memory each time: Bakta 6ย GB, then 12ย GB, then 18ย GB.
    2. Never past --max_memory: a request above it is lowered to it.
    3. Still failing after that: the run stops. Raise --max_memory, or the memory Docker Desktop has.

    Any other exit, in a step run per sample

    1. Tried again, 2 more times.
    2. Still failing: that sample's task is set aside, and the run carries on with the others.
    3. Its work directory, named in the log, says why.

    The steps run once for all the samples are retried too, and then stop the run, except the pangenome: like a step run per sample, it is set aside after two more tries, and the reports go on without it. Some stop it at once: VALIDATE_INPUTS on any failure, and the eggNOG annotation on anything but memory or time.

    What nextflow.config does with a task that fails.
    +

    Before the pipeline starts

    #
    +

    "Unexpected input: '('" in nextflow.config

    #
    +

    Nextflowย 25.10 and later read pipelines with a stricter parser, which older copies of AMAP do not pass. Nextflow keeps the copy it downloaded first, so fetch the current one with nextflow pull PathoGenOmics-Lab/AMAP and run again. To run an older AMAP release as it is, export NXF_SYNTAX_PARSER=v1 first.

    +

    "Input TSV file not found"

    #
    +

    The path in --tsv is wrong, or relative to somewhere other than where you launched Nextflow.

    +

    "Invalid TSV format"

    #
    +

    The header must be exactly sampleId and fasta, separated by a real tab. Spaces are the usual culprit. cat -t samples.tsv shows tabs as ^I, and works on both Linux and macOS.

    +

    "FASTA file not found"

    #
    +

    A path in the sample sheet does not resolve. Absolute paths avoid the ambiguity.

    +

    "not found, neither on the host nor inside the container"

    #
    +

    A reference file the run reads is in neither place: a typo in its path, or one of the defaults, which ship in the container, on a run without one. Give the path to your own copy with its reference argument.

    +

    Validation failures

    #
    +

    A check that fails stops the run, and Nextflow prints the validator's whole report, which check failed and for which sample, under Command output, followed by the task's Work dir. Only a check that passes writes results/pipeline_info/validation.log; after a failure the log is in that work folder. With --strict_validation, a warning fails the same way. Thresholds are in preparing inputs.

    +

    Container problems

    #
    +

    "Cannot connect to the Docker daemon"

    #
    +

    Docker is installed but not running: start Docker Desktop, or the Docker service on Linux. docker info answers once it is up, where docker --version answers either way.

    +

    "permission denied while trying to connect to the Docker daemon socket"

    #
    +

    Your user is not in the docker group. Either add it, or use Singularity.

    +

    The image will not pull

    #
    +

    Check network access from the compute nodes, which often differs from the login node. Pull the image once on the login node, into a folder every node sees, and have the runs look there. The singularity profile keeps its images in <outdir>/singularity_cache, and that setting wins over NXF_SINGULARITY_CACHEDIR, so the variable alone changes nothing; set the folder in a config passed with -c instead:

    +
    shared.config
    singularity.cacheDir = '/shared/singularity_cache'
    +
    Terminal
    $ singularity pull /shared/singularity_cache/paururo-amap-latest.img \
+    docker://paururo/amap:latest
+$ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
+    -c shared.config --tsv samples.tsv
    +

    paururo-amap-latest.img is the name Nextflow gives that image in its cache, so the run finds it there and pulls nothing.

    +

    Slow on Apple Silicon

    #
    +

    The image is amd64 only, so it emulates. Fine for the test profile, painful beyond that. Use an amd64 machine.

    +

    Memory

    #
    +

    Killed with exit 137, 140 or 143

    #
    +

    Exit 137 is the out-of-memory killer, 140 a time limit and 143 a SIGTERM, the way a scheduler usually ends a job at its limits. AMAP retries a task that exits with any of them, asking for more memory on each attempt, so a single occurrence in the log that later succeeds is normal. Persistent failures mean raising the ceiling, with --max_memory 256.GB for instance.

    +

    On a cluster, that is the whole answer. On a macOS or Windows laptop it is the wrong one: Docker Desktop runs containers inside a VM with its own memory ceiling, commonly 8ย GB, and that ceiling is what killed the process. Nextflow sized the request against the host and never saw it. Raising --max_memory only makes the request larger than a limit it was already over. Raise the VM in Docker Desktop under Settings, Resources, Memory, then keep --max_memory at or below it.

    +

    eggNOG runs out of memory

    #
    +

    It is the hungriest step, requesting 50ย GB, and it runs once over all samples together. On a machine that cannot reach that, --skip_functional_annotation drops it along with the pangenome, the duplication analysis and final_report.html, and still gives you the corrected genes and amap_report.html. On a cluster, use the highmem profile, or point it at a large-memory partition:

    +
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm,highmem \
+    --tsv samples.tsv
    +

    Filesystem

    #
    +

    "No space left on device"

    #
    +

    Usually the work directory, which grows to tens of gigabytes on a large run. Move it somewhere with room by adding -work-dir /scratch/$USER/amap_work to the command. Also check your $TMPDIR quota, since Diamond and Bakta both use it heavily, and the folder --eggnog_db names: eggnog.db alone is 41.4ย GB.

    +

    Everything is slow on aย shared filesystem

    #
    +

    Confirm --scratch is on. It defaults to true in the executor profiles, but a standard or docker profile on a cluster will not have it.

    +

    Cache misses on resume

    #
    +

    NFS timestamp drift across nodes invalidates Nextflow's cache. The slurm profile already sets process.cache = 'lenient' for this. With another executor, add -process.cache lenient to the command.

    +

    The eggNOG database

    #
    +

    It downloads every run

    #
    +

    --eggnog_db is pointing somewhere per-run. Give it persistent shared storage.

    +

    It rebuilds itself unexpectedly

    #
    +

    A bacteria.dmnd under 100ย MB is treated as a truncated download and rebuilt. Usually means an earlier download was interrupted.

    +

    "has eggnog_db.ok, so EGGNOG_DB will not run again"

    #
    +

    eggnog_db.ok tells Nextflow the database is built, so it never runs EGGNOG_DB again, but what it needs is not all there. Delete the files the message names and the next run builds it again.

    +

    Results that look wrong

    #
    +

    Far fewer genes than expected

    #
    +

    Check the assembly first: *-AMAP.qc.tsv has the contig count and the N50. A gene that a contig end cuts is not lost: it comes back in pieces, each under the gene's name and marked partial at its open end: partial in the master table, the Partial state in amap_report.html. Only a piece under 8ย residues is too short to place: in CDC1551 cut into 40 contigs, 22 of the 25 smaller pieces were placed, and the other three were five or six codons long (pieces of cut genes).

    +

    *-rvmissing.tsv is a different list: the reference genes Bakta missed and AMAP put back.

    +

    "genes lost to (Start, End, Strand, Contig) collisions"

    #
    +

    Two of the classification's sources listed a gene at exactly the same place, and one entry was kept. Genes on different contigs never collide, since the contig is part of the key.

    +

    A sample is missing from the results ("Ignored : 1")

    #
    +

    A task run per sample that fails for any reason but its limits (exit 137, 140 or 143) is tried twice more and then set aside, so one bad genome does not stop the others. The run still ends โœ“ COMPLETED, its summary counts the task under Ignored, and a warning after it says how many failed. The genome's results stop at that step: set aside at the correction or before it, it has no folder under AMAP/, and if it was the correction itself, amap_report.html names it at the top. To find which task it was and why:

    +
    Terminal
    $ grep -w FAILED results/pipeline_info/execution_trace_*.txt
    +

    The trace's second column is the task's hash, the start of its folder under work/, and that folder's .command.err says why. A dot in a sample ID is one cause: see preparing inputs.

    +

    The clean proteome has fewer sequences than the GFF3 has genes

    #
    +

    Expected. *-AMAP.clean.faa drops pseudogenes and internal-stop sequences. Use *-AMAP.faa if you need one protein per annotated gene.

    +

    Almost everything is caseย 1

    #
    +

    Your samples are too far from the reference for the alignments to find anything, so there is nothing to correct against. Use a closer reference. See custom reference.

    +

    The Mycobrowser sections are empty

    #
    +

    Expected with --skip_mycobrowser, or when the Mycobrowser file is missing. The pipeline warns and continues.

    +

    Still stuck

    #

    Open an issue at github.com/PathoGenOmics-Lab/AMAP/issues with:

    -
    • the command you ran
    • the failing task's .command.err
    • results/pipeline_info/software_versions.yml
    • results/pipeline_info/params.yml
    +
    • the command you ran
    • the failing task's .command.err
    • results/pipeline_info/software_versions.yml
    • results/pipeline_info/params.yml

    The last two pin down the exact versions and parameters, which is usually what the answer turns on.

    -

    Edit this page on GitHub

    - +

    Back to the top

    +

    Edit this page on GitHub

    +
    - - diff --git a/docs/deployment/troubleshooting.md b/docs/deployment/troubleshooting.md index 3252bde..cdfda1b 100644 --- a/docs/deployment/troubleshooting.md +++ b/docs/deployment/troubleshooting.md @@ -1,5 +1,8 @@ # Troubleshooting +Each problem below is a heading, with the message the pipeline or the system +prints in quotes where there is one, so you can search for what you see. + ## Reading a failure When a process fails, Nextflow prints the work directory for the failed task. @@ -24,51 +27,95 @@ resources or logic. ## Before the pipeline starts -- **"Unexpected input: '('" in `nextflow.config`**: Nextflow 25.10 and later read pipelines with a stricter parser, which older - copies of AMAP do not pass. Nextflow keeps the copy it downloaded first, so - fetch the current one with `nextflow pull PathoGenOmics-Lab/AMAP` and run - again. To run an older AMAP release as it is, `export NXF_SYNTAX_PARSER=v1` - first. -- **"Input TSV file not found"**: The path in `--tsv` is wrong, or relative to somewhere other than where you - launched Nextflow. -- **"Invalid TSV format"**: The header must be exactly `sampleId` and `fasta`, separated by a real tab. - Spaces are the usual culprit. `cat -t samples.tsv` shows tabs as `^I`, and - works on both Linux and macOS. -- **"FASTA file not found"**: A path in the sample sheet does not resolve. Absolute paths avoid the - ambiguity. -- **"not found, neither on the host nor inside the container"**: A reference file the run reads is in neither place: a typo in its path, or - one of the defaults, which ship in the container, on a run without one. Give - the path to your own copy with its - [reference argument](../user-guide/parameters.md#reference-and-database-arguments). -- **Validation failures**: `results/pipeline_info/validation.log` says which check failed and for which - sample. Thresholds are in [preparing inputs](../getting-started/inputs.md). +### "Unexpected input: '('" in `nextflow.config` + +Nextflow 25.10 and later read pipelines with a stricter parser, which older +copies of AMAP do not pass. Nextflow keeps the copy it downloaded first, so +fetch the current one with `nextflow pull PathoGenOmics-Lab/AMAP` and run +again. To run an older AMAP release as it is, `export NXF_SYNTAX_PARSER=v1` +first. + +### "Input TSV file not found" + +The path in `--tsv` is wrong, or relative to somewhere other than where you +launched Nextflow. + +### "Invalid TSV format" + +The header must be exactly `sampleId` and `fasta`, separated by a real tab. +Spaces are the usual culprit. `cat -t samples.tsv` shows tabs as `^I`, and +works on both Linux and macOS. + +### "FASTA file not found" + +A path in the sample sheet does not resolve. Absolute paths avoid the +ambiguity. + +### "not found, neither on the host nor inside the container" + +A reference file the run reads is in neither place: a typo in its path, or one +of the defaults, which ship in the container, on a run without one. Give the +path to your own copy with its +[reference argument](../user-guide/parameters.md#reference-and-database-arguments). + +### Validation failures + +A check that fails stops the run, and Nextflow prints the validator's whole +report, which check failed and for which sample, under `Command output`, +followed by the task's `Work dir`. Only a check that passes writes +`results/pipeline_info/validation.log`; after a failure the log is in that work +folder. With `--strict_validation`, a warning fails the same way. Thresholds +are in [preparing inputs](../getting-started/inputs.md). ## Container problems -**The image will not pull.** Check network access from the compute nodes, which -often differs from the login node. Pre-pulling on the login node and setting -`NXF_SINGULARITY_CACHEDIR` sidesteps it: +### "Cannot connect to the Docker daemon" + +Docker is installed but not running: start Docker Desktop, or the Docker +service on Linux. `docker info` answers once it is up, where `docker --version` +answers either way. + +### "permission denied while trying to connect to the Docker daemon socket" + +Your user is not in the `docker` group. Either add it, or use Singularity. + +### The image will not pull + +Check network access from the compute nodes, which often differs from the login +node. Pull the image once on the login node, into a folder every node sees, and +have the runs look there. The `singularity` profile keeps its images in +`/singularity_cache`, and that setting wins over +`NXF_SINGULARITY_CACHEDIR`, so the variable alone changes nothing; set the +folder in a config passed with `-c` instead: + +```groovy title="shared.config" +singularity.cacheDir = '/shared/singularity_cache' +``` ```bash -export NXF_SINGULARITY_CACHEDIR=/shared/singularity_cache -singularity pull docker://paururo/amap:latest +singularity pull /shared/singularity_cache/paururo-amap-latest.img \ + docker://paururo/amap:latest +nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \ + -c shared.config --tsv samples.tsv ``` -**Slow on Apple Silicon.** The image is amd64 only, so it emulates. Fine for the -test profile, painful beyond that. Use an amd64 machine. +`paururo-amap-latest.img` is the name Nextflow gives that image in its cache, +so the run finds it there and pulls nothing. -**Docker permission denied.** Your user is not in the `docker` group. Either add -it, or use Singularity. +### Slow on Apple Silicon + +The image is amd64 only, so it emulates. Fine for the test profile, painful +beyond that. Use an amd64 machine. ## Memory -**A process is killed with exit 137 or 140.** Out of memory. AMAP retries with -more, scaling by attempt, so a single occurrence in the log that later succeeds -is normal. Persistent failures mean raising the ceiling: +### Killed with exit 137, 140 or 143 -```bash ---max_memory 256.GB -``` +Exit 137 is the out-of-memory killer, 140 a time limit and 143 a SIGTERM, the +way a scheduler usually ends a job at its limits. AMAP retries a task that +exits with any of them, asking for more memory on each attempt, so a single +occurrence in the log that later succeeds is normal. Persistent failures mean raising the ceiling, with +`--max_memory 256.GB` for instance. On a cluster, that is the whole answer. On a macOS or Windows laptop it is the wrong one: Docker Desktop runs containers inside a VM with its own memory @@ -78,11 +125,14 @@ only makes the request larger than a limit it was already over. Raise the VM in Docker Desktop under Settings, Resources, Memory, then keep `--max_memory` at or below it. -**eggNOG runs out of memory.** It is the hungriest step, requesting 50 GB, and -it runs once over all samples together. On a machine that cannot reach that, -`--skip_functional_annotation` drops it along with the pangenome and the -duplication analysis and still gives you the corrected annotation. On a cluster, -use the `highmem` profile, or point it at a large-memory partition: +### eggNOG runs out of memory + +It is the hungriest step, requesting 50 GB, and it runs once over all samples +together. On a machine that cannot reach that, `--skip_functional_annotation` +drops it along with the pangenome, the duplication analysis and +`final_report.html`, and still gives you the corrected genes and +`amap_report.html`. On a cluster, use the `highmem` profile, or point it at a +large-memory partition: ```bash nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm,highmem \ @@ -91,62 +141,95 @@ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm,highmem \ ## Filesystem -**"No space left on device".** Usually the work directory, which grows to tens -of gigabytes. Move it somewhere with room: +### "No space left on device" -```bash --work-dir /scratch/$USER/amap_work -``` +Usually the work directory, which grows to tens of gigabytes on a large run. +Move it somewhere with room by adding `-work-dir /scratch/$USER/amap_work` to +the command. Also check your `$TMPDIR` quota, since Diamond and Bakta both use +it heavily, and the folder `--eggnog_db` names: `eggnog.db` alone is 41.4 GB. -Also check your `$TMPDIR` quota, since Diamond and Bakta both use it heavily. +### Everything is slow on a shared filesystem -**Everything is slow on a shared filesystem.** Confirm `--scratch` is on. It -defaults to true in the HPC profiles, but a `standard` or `docker` profile on a -cluster will not have it. +Confirm `--scratch` is on. It defaults to true in the executor profiles, but a +`standard` or `docker` profile on a cluster will not have it. -**Cache misses on resume.** NFS timestamp drift across nodes invalidates -Nextflow's cache. The `slurm` profile already sets `process.cache = 'lenient'` -for this. If you are using a different executor, add it: +### Cache misses on resume -```bash --process.cache lenient -``` +NFS timestamp drift across nodes invalidates Nextflow's cache. The `slurm` +profile already sets `process.cache = 'lenient'` for this. With another +executor, add `-process.cache lenient` to the command. ## The eggNOG database -**It downloads every run.** `--eggnog_db` is pointing somewhere per-run. Give it -persistent shared storage. +### It downloads every run -**It rebuilds itself unexpectedly.** A `bacteria.dmnd` under 100 MB is treated as -a truncated download and rebuilt. Usually means an earlier download was -interrupted. +`--eggnog_db` is pointing somewhere per-run. Give it persistent shared storage. -**"has eggnog_db.ok, so EGGNOG_DB will not run again".** `eggnog_db.ok` tells -Nextflow the database is built, so it never runs `EGGNOG_DB` again, but what it -needs is not all there. Delete the files the message names and the next run -builds it again. +### It rebuilds itself unexpectedly + +A `bacteria.dmnd` under 100 MB is treated as a truncated download and rebuilt. +Usually means an earlier download was interrupted. + +### "has eggnog_db.ok, so EGGNOG_DB will not run again" + +`eggnog_db.ok` tells Nextflow the database is built, so it never runs +`EGGNOG_DB` again, but what it needs is not all there. Delete the files the +message names and the next run builds it again. ## Results that look wrong -**Far fewer genes than expected.** Check the assembly. A fragmented assembly -loses genes across contig boundaries, and they turn up in `*-rvmissing.tsv` -rather than being corrected. `*-AMAP.qc.tsv` has the contig count and N50. +### Far fewer genes than expected + +Check the assembly first: `*-AMAP.qc.tsv` has the contig count and the N50. A +gene that a contig end cuts is not lost: it comes back in pieces, each under the +gene's name and marked partial at its open end: `partial` in the master table, +the Partial state in `amap_report.html`. Only a piece under 8 residues is too +short to place: in CDC1551 cut into 40 contigs, 22 of the 25 smaller pieces +were placed, and the other three were five or six codons long +([pieces of cut genes](../how-it-works/correction-algorithm.md#pieces-of-cut-genes)). + +`*-rvmissing.tsv` is a different list: the reference genes Bakta missed and AMAP +put back. -**"Genes lost to (Start, End, Strand, Contig) collisions" in the log.** Two of -the classification's sources listed a gene at exactly the same place, and one -entry was kept. Genes on different contigs never collide, since the contig is -part of the key. +### "genes lost to (Start, End, Strand, Contig) collisions" -**`*-AMAP.clean.faa` has fewer sequences than the GFF3 has genes.** Expected. -The clean proteome drops pseudogenes and internal-stop sequences. Use +Two of the classification's sources listed a gene at exactly the same place, +and one entry was kept. Genes on different contigs never collide, since the +contig is part of the key. + +### A sample is missing from the results ("Ignored : 1") + +A task run per sample that fails for any reason but its limits (exit 137, 140 +or 143) is tried twice more and then set aside, so one bad genome does not stop +the others. The run still ends `โœ“ COMPLETED`, its summary counts the task +under `Ignored`, and a warning after it says how many failed. The genome's +results stop at that step: set aside at the correction or before it, it has no +folder under `AMAP/`, and if it was the correction itself, `amap_report.html` +names it at the top. To find which task it was and why: + +```bash +grep -w FAILED results/pipeline_info/execution_trace_*.txt +``` + +The trace's second column is the task's hash, the start of its folder under +`work/`, and that folder's `.command.err` says why. A dot in a sample ID is one +cause: see [preparing inputs](../getting-started/inputs.md#the-sample-sheet). + +### The clean proteome has fewer sequences than the GFF3 has genes + +Expected. `*-AMAP.clean.faa` drops pseudogenes and internal-stop sequences. Use `*-AMAP.faa` if you need one protein per annotated gene. -**Almost everything is case 1.** Your samples are too far from the reference for -the alignments to find anything, so there is nothing to correct against. Use a -closer reference. See [custom reference](../user-guide/custom-reference.md). +### Almost everything is case 1 + +Your samples are too far from the reference for the alignments to find +anything, so there is nothing to correct against. Use a closer reference. See +[custom reference](../user-guide/custom-reference.md). + +### The Mycobrowser sections are empty -**The Mycobrowser sections are empty.** Expected with `--skip_mycobrowser`, or -when the Mycobrowser file is missing. The pipeline warns and continues. +Expected with `--skip_mycobrowser`, or when the Mycobrowser file is missing. +The pipeline warns and continues. ## Still stuck diff --git a/docs/documentation.html b/docs/documentation.html index e246cee..4e8187e 100644 --- a/docs/documentation.html +++ b/docs/documentation.html @@ -4,109 +4,114 @@ Documentation | AMAP - + + - + - + - + - +
    - AMAP + AMAP
    - + + +

    Documentation

    -

    The long versions: every option, every file, and how each decision is made. New to AMAP? Start here, then follow the tutorial.

    -

    Getting started

    +

    The long versions: every option, every file, and how each decision is made. New to AMAP? Start here, then follow the tutorial.

    + +

    Getting started

    -TutorialFrom an empty folder to a corrected genome, with the numbers of a real run. InstallationNextflow, a container engine, and running without one. -QuickstartFrom an installed Nextflow to an open report. +QuickstartFrom an installed Nextflow to an open report: the test, a sample sheet, the check, the run and the results. +TutorialOne real genome, start to finish, with every number. Preparing inputsThe sample sheet, the genomes, and the checks run on them.
    -

    User guide

    +

    User guide

    ParametersEvery option, its default, and when to change it. Running AMAPProfiles, schedulers and larger runs. OutputsEvery folder and file, and what a healthy run looks like. -The reportWhat each section of final_report.html shows. -The interactive reportamap_report.html, question by question, with or without eggNOG. +The full reportEach section of the full report, and what it shows. +The report by questionEvery question the report answers, with or without eggNOG. Custom referenceAny organism, against a trusted annotation of your own.
    -

    How it works

    +

    How it works

    Step by step, illustratedOne gene at a time, from Bakta's call to what AMAP keeps. OverviewWhat goes into a decision, and the five outcomes. @@ -114,13 +119,13 @@

    How it works

    The correction algorithmEvery rule and threshold, drawn on real genes. Duplication metricsWhat the network numbers mean.
    -

    Deployment

    +

    Deployment

    ContainersThe AMAP image, what is in it, and building your own. HPC clustersSchedulers, scratch, and where things should live. TroubleshootingOrganised by the error you are looking at.
    -

    About

    +

    About

    ChangelogWhat changed in each version. CitationHow to cite AMAP and the tools it runs. @@ -132,7 +137,7 @@

    About

    Cite

    -
    Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software].
    +
    Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software].

    Software: doi:10.5281/zenodo.8024446

    @@ -142,7 +147,7 @@

    Learn more

  • Documentation
  • How it works
  • Parameters
    • -
  • Changelog
    • +
  • Changelog
    • @@ -151,12 +156,11 @@

    Project

  • GitHub
  • Report a bug
  • Docker image
    • -
  • GPL-3.0 · PathoGenOmics Lab
    • +
    PathoGenOmics LabGPL-3.0AMAP 1.0.1
    - diff --git a/docs/examples/demo_report.html b/docs/examples/demo_report.html index fc6ccc2..92599f2 100644 --- a/docs/examples/demo_report.html +++ b/docs/examples/demo_report.html @@ -1,6 +1,6 @@ - + @@ -4795,6 +4795,7 @@ div.column{display: inline-block; vertical-align: top; width: 50%;} div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;} ul.task-list{list-style: none;} +.display.math{display: block; text-align: center; margin: 0.5rem auto;} @@ -4879,10 +4880,11 @@ background:var(--amap-teal-dark); border-radius:7px 7px 0 0; } + body #TOC .tocify-item.active, body #TOC .list-group-item.active{ -background:var(--amap-teal)!important; -border-color:var(--amap-teal)!important; +background:var(--amap-teal-dark)!important; +border-color:var(--amap-teal-dark)!important; color:#fff!important; } body #TOC .list-group-item:hover{ @@ -5114,6 +5116,13 @@ PathoGenOmics Lab + + + + @@ -5142,6 +5151,7 @@

    18/10/2026

    overlay.className = "zoomDiv"; var big = document.createElement("img"); big.className = "zoomImg"; + big.alt = ""; overlay.appendChild(big); document.body.insertBefore(overlay, document.body.firstChild); @@ -5155,6 +5165,7 @@

    18/10/2026

    if (t.tagName !== "IMG") return; if (t.classList.contains("zoomImg")) { close(); return; } big.src = t.getAttribute("src"); + big.alt = t.getAttribute("alt") || ""; big.style.width = "100%"; Object.assign(overlay.style, { opacity: "1", width: "auto", border: "1px solid white", @@ -5191,14 +5202,14 @@

    Overview

    Dataset Composition

    This section provides a general overview of the dataset, showing the distribution of sequences across lineages/groups.

    -

    +

    Bar chart of how many genomes each lineage has, next to a ring of each lineage's share of the genomes.

    Sample Quality Overview

    Per-sample summary of total CDS, correction rates, and missing reference genes.

    -
    - +
    +

    Annotation Quality Metrics

    @@ -5206,20 +5217,20 @@

    Annotation Quality Metrics

    density, hypothetical %, and functional annotation rate.

    Overview Table

    -
    - +
    +

    Coding Density

    -

    +

    Bar chart of each sample's coding density, in percent, with the mean as a dashed red line.

    Hypothetical %

    -

    +

    Bar chart of each sample's share of hypothetical proteins, in percent, with the mean as a dashed red line.

    Genome Size Distribution

    -

    +

    Histogram of the genome sizes in megabases, with the mean as a dashed red line.

    @@ -5227,15 +5238,15 @@

    Genome Size Distribution

    AMAP Annotation Results

    CDS Distribution per Lineage

    -

    +

    Box plot of the number of CDS per genome in each lineage, each genome a point.

    CDS not in Reference

    -

    +

    Box plot of how many CDS of each genome name no reference gene, by lineage, each genome a point.

    CDS Correction Summary

    -

    +

    Stacked bars, one per lineage, splitting its CDS into those that matched the reference and those AMAP corrected, in percent.

    Missing Reference Genes (Rv)

    @@ -5245,12 +5256,12 @@

    Missing Reference Genes (Rv)

    Heatmap

    Which Rv genes are missing in which lineages? Darker = missing in more genomes of that lineage.

    -

    +

    Horizontal bars for the 50 reference genes missing from the most genomes, coloured by lineage, each bar as long as the number of genomes missing that gene.

    Full Table

    -
    - +
    +
    @@ -5260,234 +5271,234 @@

    Functional Annotation

    COG Categories (EggNOG)

    Overview

    -
    - +
    +

    Statistics

    -
    - +
    +

    Per COG (% Percentage)

    A

    -

    +

    Box plot of the percentage of CDS in COG category A - RNA processing and modification, per genome and by lineage, with Dunn's tests between lineages.

    B

    -

    +

    Box plot of the percentage of CDS in COG category B - Chromatin structure and dynamics, per genome and by lineage, with Dunn's tests between lineages.

    C

    -

    +

    Box plot of the percentage of CDS in COG category C - Energy production and conversion, per genome and by lineage, with Dunn's tests between lineages.

    D

    -

    +

    Box plot of the percentage of CDS in COG category D - Cell cycle control, cell division, per genome and by lineage, with Dunn's tests between lineages.

    E

    -

    +

    Box plot of the percentage of CDS in COG category E - Amino acid transport and metabolism, per genome and by lineage, with Dunn's tests between lineages.

    F

    -

    +

    Box plot of the percentage of CDS in COG category F - Nucleotide transport and metabolism, per genome and by lineage, with Dunn's tests between lineages.

    G

    -

    +

    Box plot of the percentage of CDS in COG category G - Carbohydrate transport and metabolism, per genome and by lineage, with Dunn's tests between lineages.

    H

    -

    +

    Box plot of the percentage of CDS in COG category H - Coenzyme transport and metabolism, per genome and by lineage, with Dunn's tests between lineages.

    I

    -

    +

    Box plot of the percentage of CDS in COG category I - Lipid transport and metabolism, per genome and by lineage, with Dunn's tests between lineages.

    J

    -

    +

    Box plot of the percentage of CDS in COG category J - Translation, ribosomal structure, per genome and by lineage, with Dunn's tests between lineages.

    K

    -

    +

    Box plot of the percentage of CDS in COG category K - Transcription, per genome and by lineage, with Dunn's tests between lineages.

    L

    -

    +

    Box plot of the percentage of CDS in COG category L - Replication, recombination and repair, per genome and by lineage, with Dunn's tests between lineages.

    M

    -

    +

    Box plot of the percentage of CDS in COG category M - Cell wall/membrane/envelope biogenesis, per genome and by lineage, with Dunn's tests between lineages.

    N

    -

    +

    Box plot of the percentage of CDS in COG category N - Cell motility, per genome and by lineage, with Dunn's tests between lineages.

    O

    -

    +

    Box plot of the percentage of CDS in COG category O - Post-translational modification, chaperones, per genome and by lineage, with Dunn's tests between lineages.

    P

    -

    +

    Box plot of the percentage of CDS in COG category P - Inorganic ion transport, per genome and by lineage, with Dunn's tests between lineages.

    Q

    -

    +

    Box plot of the percentage of CDS in COG category Q - Secondary metabolites biosynthesis, per genome and by lineage, with Dunn's tests between lineages.

    R

    -

    +

    Box plot of the percentage of CDS in COG category R - General function prediction only, per genome and by lineage, with Dunn's tests between lineages.

    S

    -

    +

    Box plot of the percentage of CDS in COG category S - Function unknown, per genome and by lineage, with Dunn's tests between lineages.

    T

    -

    +

    Box plot of the percentage of CDS in COG category T - Signal transduction mechanisms, per genome and by lineage, with Dunn's tests between lineages.

    U

    -

    +

    Box plot of the percentage of CDS in COG category U - Intracellular trafficking, secretion, per genome and by lineage, with Dunn's tests between lineages.

    V

    -

    +

    Box plot of the percentage of CDS in COG category V - Defense mechanisms, per genome and by lineage, with Dunn's tests between lineages.

    W

    -

    +

    Box plot of the percentage of CDS in COG category W - Extracellular structures, per genome and by lineage, with Dunn's tests between lineages.

    X

    -

    +

    Box plot of the percentage of CDS in COG category X - Mobilome: prophages, transposons, per genome and by lineage, with Dunn's tests between lineages.

    Y

    -

    +

    Box plot of the percentage of CDS in COG category Y - Nuclear structure, per genome and by lineage, with Dunn's tests between lineages.

    Z

    -

    +

    Box plot of the percentage of CDS in COG category Z - Cytoskeleton, per genome and by lineage, with Dunn's tests between lineages.

    NA

    -

    +

    Box plot of the percentage of CDS in COG category None, per genome and by lineage, with Dunn's tests between lineages.

    Per COG (Nยบ CDS)

    A

    -

    +

    Box plot of the number of CDS in COG category A - RNA processing and modification, per genome and by lineage, with Dunn's tests between lineages.

    B

    -

    +

    Box plot of the number of CDS in COG category B - Chromatin structure and dynamics, per genome and by lineage, with Dunn's tests between lineages.

    C

    -

    +

    Box plot of the number of CDS in COG category C - Energy production and conversion, per genome and by lineage, with Dunn's tests between lineages.

    D

    -

    +

    Box plot of the number of CDS in COG category D - Cell cycle control, cell division, per genome and by lineage, with Dunn's tests between lineages.

    E

    -

    +

    Box plot of the number of CDS in COG category E - Amino acid transport and metabolism, per genome and by lineage, with Dunn's tests between lineages.

    F

    -

    +

    Box plot of the number of CDS in COG category F - Nucleotide transport and metabolism, per genome and by lineage, with Dunn's tests between lineages.

    G

    -

    +

    Box plot of the number of CDS in COG category G - Carbohydrate transport and metabolism, per genome and by lineage, with Dunn's tests between lineages.

    H

    -

    +

    Box plot of the number of CDS in COG category H - Coenzyme transport and metabolism, per genome and by lineage, with Dunn's tests between lineages.

    I

    -

    +

    Box plot of the number of CDS in COG category I - Lipid transport and metabolism, per genome and by lineage, with Dunn's tests between lineages.

    J

    -

    +

    Box plot of the number of CDS in COG category J - Translation, ribosomal structure, per genome and by lineage, with Dunn's tests between lineages.

    K

    -

    +

    Box plot of the number of CDS in COG category K - Transcription, per genome and by lineage, with Dunn's tests between lineages.

    L

    -

    +

    Box plot of the number of CDS in COG category L - Replication, recombination and repair, per genome and by lineage, with Dunn's tests between lineages.

    M

    -

    +

    Box plot of the number of CDS in COG category M - Cell wall/membrane/envelope biogenesis, per genome and by lineage, with Dunn's tests between lineages.

    N

    -

    +

    Box plot of the number of CDS in COG category N - Cell motility, per genome and by lineage, with Dunn's tests between lineages.

    O

    -

    +

    Box plot of the number of CDS in COG category O - Post-translational modification, chaperones, per genome and by lineage, with Dunn's tests between lineages.

    P

    -

    +

    Box plot of the number of CDS in COG category P - Inorganic ion transport, per genome and by lineage, with Dunn's tests between lineages.

    Q

    -

    +

    Box plot of the number of CDS in COG category Q - Secondary metabolites biosynthesis, per genome and by lineage, with Dunn's tests between lineages.

    R

    -

    +

    Box plot of the number of CDS in COG category R - General function prediction only, per genome and by lineage, with Dunn's tests between lineages.

    S

    -

    +

    Box plot of the number of CDS in COG category S - Function unknown, per genome and by lineage, with Dunn's tests between lineages.

    T

    -

    +

    Box plot of the number of CDS in COG category T - Signal transduction mechanisms, per genome and by lineage, with Dunn's tests between lineages.

    U

    -

    +

    Box plot of the number of CDS in COG category U - Intracellular trafficking, secretion, per genome and by lineage, with Dunn's tests between lineages.

    V

    -

    +

    Box plot of the number of CDS in COG category V - Defense mechanisms, per genome and by lineage, with Dunn's tests between lineages.

    W

    -

    +

    Box plot of the number of CDS in COG category W - Extracellular structures, per genome and by lineage, with Dunn's tests between lineages.

    X

    -

    +

    Box plot of the number of CDS in COG category X - Mobilome: prophages, transposons, per genome and by lineage, with Dunn's tests between lineages.

    Y

    -

    +

    Box plot of the number of CDS in COG category Y - Nuclear structure, per genome and by lineage, with Dunn's tests between lineages.

    Z

    -

    +

    Box plot of the number of CDS in COG category Z - Cytoskeleton, per genome and by lineage, with Dunn's tests between lineages.

    NA

    -

    +

    Box plot of the number of CDS in COG category None, per genome and by lineage, with Dunn's tests between lineages.

    @@ -5498,91 +5509,91 @@

    Mycobrowser Categories

    is available.

    Overview

    -
    - +
    +

    Per Category (%)

    -
    -

    intermediary metabolism and respiration

    -

    +
    +

    cell wall and cell processes

    +

    Box plot of the percentage of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    conserved hypotheticals

    -

    +

    Box plot of the percentage of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    -
    -

    cell wall and cell processes

    -

    +
    +

    information pathways

    +

    Box plot of the percentage of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    +
    +
    +

    insertion seqs and phages

    +

    Box plot of the percentage of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    +
    +
    +

    intermediary metabolism and respiration

    +

    Box plot of the percentage of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    lipid metabolism

    -

    +

    Box plot of the percentage of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    -
    -

    information pathways

    -

    +
    +

    PE/PPE

    +

    Box plot of the percentage of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    regulatory proteins

    -

    -
    -
    -

    PE/PPE

    -

    +

    Box plot of the percentage of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    virulence, detoxification, adaptation

    -

    -
    -
    -

    insertion seqs and phages

    -

    +

    Box plot of the percentage of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    Per Category (Nยบ CDS)

    -
    -

    intermediary metabolism and respiration

    -

    +
    +

    cell wall and cell processes

    +

    Box plot of the number of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    conserved hypotheticals

    -

    +

    Box plot of the number of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    -
    -

    cell wall and cell processes

    -

    +
    +

    information pathways

    +

    Box plot of the number of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    +
    +
    +

    insertion seqs and phages

    +

    Box plot of the number of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    +
    +
    +

    intermediary metabolism and respiration

    +

    Box plot of the number of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    lipid metabolism

    -

    +

    Box plot of the number of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    -
    -

    information pathways

    -

    +
    +

    PE/PPE

    +

    Box plot of the number of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    regulatory proteins

    -

    -
    -
    -

    PE/PPE

    -

    +

    Box plot of the number of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    virulence, detoxification, adaptation

    -

    -
    -
    -

    insertion seqs and phages

    -

    +

    Box plot of the number of CDS in the Mycobrowser category named in its title, per genome and by lineage, with Dunn's tests between lineages.

    Statistics

    -
    - +
    +
    @@ -5592,52 +5603,52 @@

    Compared to Reference

    gains or losses in functional categories.

    Overview

    -
    - +
    +

    Per Category

    cell wall and cell processes

    -

    +

    Box plot of each genome's share of the Mycobrowser category named in its title, as a percentage of the median of all samples, by lineage.

    conserved hypotheticals

    -

    +

    Box plot of each genome's share of the Mycobrowser category named in its title, as a percentage of the median of all samples, by lineage.

    information pathways

    -

    +

    Box plot of each genome's share of the Mycobrowser category named in its title, as a percentage of the median of all samples, by lineage.

    insertion seqs and phages

    -

    +

    Box plot of each genome's share of the Mycobrowser category named in its title, as a percentage of the median of all samples, by lineage.

    intermediary metabolism and respiration

    -

    +

    Box plot of each genome's share of the Mycobrowser category named in its title, as a percentage of the median of all samples, by lineage.

    lipid metabolism

    -

    +

    Box plot of each genome's share of the Mycobrowser category named in its title, as a percentage of the median of all samples, by lineage.

    PE/PPE

    -

    +

    Box plot of each genome's share of the Mycobrowser category named in its title, as a percentage of the median of all samples, by lineage.

    regulatory proteins

    -

    +

    Box plot of each genome's share of the Mycobrowser category named in its title, as a percentage of the median of all samples, by lineage.

    virulence, detoxification, adaptation

    -

    +

    Box plot of each genome's share of the Mycobrowser category named in its title, as a percentage of the median of all samples, by lineage.

    Statistics

    -
    - +
    +
    @@ -5647,12 +5658,12 @@

    PE/PPE Gene Families

    available.

    Distribution per Lineage

    -

    +

    Bar chart of the share of each lineage's CDS that are PE/PPE genes, with how many they are.

    Statistics

    -
    - +
    +
    @@ -5660,21 +5671,33 @@

    KEGG Pathway Analysis

    Distribution of KEGG pathway annotations across lineages.

    Top Pathways

    -

    +

    Horizontal bars for the 25 KEGG pathways with the most genes, one bar per lineage.

    Full Table

    -
    - +
    +

    Pseudogene Detection

    -

    Genes without a stop codon in their annotated coordinates โ€” potential -pseudogenes.

    -
    -

    Per Lineage

    -

    +

    The genes AMAP calls pseudogenes in each sample, by lineage: internal +stop codons, a frameshift or lost stop that makes the frame run on past +the end of the reference protein, no stop codon within reach, or a stop +the reference itself lacks. They are the genes the GFF3 marks +pseudo=true and the QC table counts, and the master tableโ€™s +note column says why each one is a pseudogene. A sample +with none counts 0. Mycobrowserโ€™s Is_Pseudogene says +whether a gene is a pseudogene in H37Rv, not in the sample, and is not +counted.

    +
    +

    By Lineage

    +

    Box plot of the pseudogenes AMAP called per sample, by lineage, each sample a point.

    +
    +
    +

    Per Sample

    +
    +
    @@ -5686,17 +5709,17 @@

    Gene Duplications

    Duplication Summary

    48 homologous community pairs detected across all genomes.

    -

    +

    Histogram of community sizes: how many communities have each number of genes.

    Cross-genome Communities

    -
    - +
    +

    Network Metrics

    -
    - +
    +
    @@ -5704,15 +5727,15 @@

    Pangenome Analysis

    Gene presence/absence across samples based on pangene analysis.

    Core vs Accessory Genes

    -

    +

    Bar chart of the pangenome's gene families split into core, soft core, shell and cloud, with counts and percentages.

    Gene Frequency Spectrum

    -

    +

    Histogram of gene families by how many samples have them, with a dashed red line at all samples, the core.

    Presence/Absence Heatmap

    -

    +

    Heatmap of which samples have each variable gene family, present in green and absent in white, rows and columns clustered.

    @@ -5721,7 +5744,7 @@

    AMAP CDS Stats

    AMAP.

    Classification:

      -
    • Case 1 keep: CDS missing from the reference โ€” +
    • Case 1 keep: CDS missing from the reference, potentially real novel genes
    • Case 1 remove: CDS falsely introduced by Bakta, absent from reference
      • @@ -5735,61 +5758,61 @@

      AMAP CDS Stats

    Boxplot

    -

    +

    Box plots of the number of CDS in each case, one panel per case and a box per lineage, with Dunn's tests between lineages.

    Violinplot

    -

    +

    Violin plots of the number of CDS in each case, one panel per case and a violin per lineage, with the mean and its confidence interval.

    Histogram Cases

    -

    +

    Stacked bars, one per lineage, of the share of its mean CDS count in each case.

    Heatmap

    -

    +

    Heatmap of the log mean number of CDS in each case, lineage by lineage.

    Per-case Statistics

    Case 1

    -
    - +
    +

    Case 1 Keep

    -
    - +
    +

    Case 1 Remove

    -
    - +
    +

    Case 2

    -
    - +
    +

    Case 3

    -
    - +
    +

    Case 4

    -
    - +
    +

    Total CDS

    -
    - +
    +

    Missing Rvs

    -
    - +
    +
    @@ -5797,11 +5820,11 @@

    Missing Rvs

    Master Table Explorer

    Interactive explorer for the full annotation master table with all functional annotations.

    -
    - +
    +

    Report generated by AMAP v1.0.1 on -2026-10-18 15:43.

    +2026-10-24 10:41.

    @@ -5870,15 +5893,6 @@

    Master Table Explorer

    }); - - diff --git a/docs/getting-started/inputs.html b/docs/getting-started/inputs.html index d576b49..2da7c8f 100644 --- a/docs/getting-started/inputs.html +++ b/docs/getting-started/inputs.html @@ -4,145 +4,167 @@ Preparing inputs | AMAP docs - + + - + - + - + - +
    - AMAP + AMAP
    - + + +
    -

    Docs / Getting started

    +

    Docs / Getting started

    Describes AMAP on main, ahead of 1.0.1: what changed

    Preparing inputs

    +

    The sample sheet, the genomes, and the checks run on them.

    +

    3 min read

    +
    On this page
    +

    AMAP takes assembled genomes. It does not do assembly, so reads are not an input here.

    -

    The sample sheet#

    +

    The sample sheet

    #

    A tab-separated file with a header row and exactly two columns:

    -
    samples.tsv
    sampleId	fasta
-G26689	/data/genomes/G26689.fasta
-G26690	/data/genomes/G26690.fasta
-L4_G12345	./assemblies/L4_G12345.fa
    -

    samples.tsv2

    1sampleId⇥3fasta
    G26689⇥/data/genomes/G26689.fasta
    G26690⇥/data/genomes/G26690.fasta
    L4_G12345⇥./assemblies/L4_G12345.fa
    1. sampleId: unique, and carried into every folder and file name. Letters, digits and underscores.
    2. A real tab between the two columns, on every line, header included. Spaces will not do.
    3. fasta: the assembled genome, as an absolute path or one relative to where you launch Nextflow.

    With spaces where the tabs go, --validate_only stops with:

    โœ— TSV missing 'sampleId' column. Found: ['sampleId    fasta']
-โœ— TSV missing 'fasta' column. Found: ['sampleId    fasta']
-โœ— Duplicate sampleIds in TSV: {''}
-โœ— Empty sampleId on line(s): [2, 3, 4]
-FAILED: 4 critical error(s)

    With tabs, and paths that do not exist yet, --validate_only stops with:

    โœ— FASTA file not found: /data/genomes/G26689.fasta
-โœ— FASTA file not found: /data/genomes/G26690.fasta
-โœ— FASTA file not found: ./assemblies/L4_G12345.fa
-โœ— FAILED: 3 critical error(s)
    The sample sheet, with its tabs drawn. Below it, what the check prints for the two mistakes it catches most: both messages come from running bin/validate_inputs.py on this sheet.
    -
    • sampleId: The name carried through every output directory and file. Must be unique. Lineage typing puts the lineage in front, so G26689 comes out as L4_G26689.
    • fasta: Path to the assembled genome. Absolute, or relative to the directory you launch Nextflow from.
    +
    samples.tsv
    sampleId	fasta
+G26689	/data/genomes/G26689.fasta
+G26690	/data/genomes/G26690.fasta
+G12345	./assemblies/G12345.fa
    +

    samples.tsv2

    1sampleId⇥3fasta
    G26689⇥/data/genomes/G26689.fasta
    G26690⇥/data/genomes/G26690.fasta
    G12345⇥./assemblies/G12345.fa
    1. sampleId: unique, and carried into every folder and file name. Letters, digits and underscores.
    2. A real tab between the two columns, on every line, header included. Spaces will not do.
    3. fasta: the assembled genome, as an absolute path or one relative to where you launch Nextflow.

    With spaces where the tabs go, nextflow run stops before any task with:

    Invalid TSV format!
+
+Expected columns: sampleId  fasta
+Found columns:    sampleId    fasta
+
+Your TSV should look like:
+  sampleId\tfasta
+  sample1\t/path/to/sample1.fasta
+  sample2\t/path/to/sample2.fasta

    With tabs, and paths that do not exist yet, nextflow run stops before any task with:

    FASTA file not found!
+
+Sample:  G26689
+Path:    /data/genomes/G26689.fasta
+
+Check that the path is correct (absolute or relative
+to where you launched nextflow).
    The sample sheet, with its tabs drawn. Below it, what a run prints for the two mistakes it catches most, --validate_only or not: both messages are main.nf's own, filled in for this sheet. They stop the run before the checks start, so no validation.log is written.
    +
    • sampleId: The name carried through every output directory and file. Must be unique. Lineage typing puts the lineage in front, so G26689 comes out as L4_G26689.
    • fasta: Path to the assembled genome. Absolute, or relative to the directory you launch Nextflow from.

    The separator has to be an actual tab. Spaces will not do, and this is the most common reason a sample sheet is rejected.

    -

    Avoid dots in sample IDs

    Several scripts derive the sample name by cutting at the first dot, so G26689.1 becomes G26689 and two such samples would collide. Stick to letters, digits and underscores.

    -

    Pre-flight validation#

    -

    VALIDATE_INPUTS runs before any computation, on every run. To run only the checks and stop:

    -
    nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --validate_only
    -

    Results are written to results/pipeline_info/validation.log.

    -

    Genome sizelog scale

    M. tuberculosis, M. bovis: 4.35 to 4.41 MbE. coli: 4.64 to 5.59 Mb
    100 kb500 kb10 Mb20 Mb
    • M. tuberculosis, M. bovis: 4.35 to 4.41 Mb
    • E. coli: 4.64 to 5.59 Mb

    GC content% of A, C, G and T

    tuberculosis complex: 65.6%E. coli: 50.5 to 50.8%
    0%20%25%50%75%80%100%
    • tuberculosis complex: 65.6%
    • E. coli: 50.5 to 50.8%

    N content% of the bases

    all six: under 0.01%
    0%1%5%10%15%
    • all six: under 0.01%

    Sequencescontigs, log scale

    closed genomes: 1Sakai: 3CDC1551 in 40 contigs
    1101005002,000
    • closed genomes: 1
    • Sakai: 3
    • CDC1551 in 40 contigs
    passeswarnswarns: highly fragmentedfails the run
    Where a genome has to sit, with the limits read off bin/validate_inputs.py, and where the six genomes of the accuracy benchmark sit: inside every range, with no warning.
    -

    What fails the run#

    -
    CheckThreshold
    Genome sizebelow 100 kb, or above 20 Mb
    N contentabove 10%
    GC contentbelow 20% or above 80%
    Empty sequencesany record with no bases
    Duplicate contig headersany repeated identifier
    Sample sheetmissing sampleId or fasta column, no data rows, duplicate or empty sample IDs
    FASTA pathfile does not exist or cannot be parsed
    Bakta databasenot found, or no version.json or bakta.db in it
    eggNOG databaseeggnog_db.ok in it while the database is incomplete, since Nextflow then never builds it again
    Reference files--ref_annotation, --ref_genome or a reference argument that the run reads but that is neither on the host nor inside the container
    -

    What only warns#

    -
    CheckThreshold
    Genome sizebelow 500 kb, or above 10 Mb
    N contentabove 1%
    GC contentbelow 25% or above 75%
    Assembly fragmentationmore than 100 contigs, and again above 500
    Short contigsany contig under 200 bp in a multi-contig assembly
    Non-standard charactersanything outside ATGCNRYSWKMBDHV
    eggNOG databasea bacteria.dmnd under 100 MB, which EGGNOG_DB takes for a truncated download and builds again
    Mycobrowser table--mycobrowser_ref neither on the host nor inside the container, so the master table goes without Mycobrowser categories
    -

    Warnings do not stop a run. Add --strict_validation to have them stop it too.

    -

    An eggNOG database that does not exist yet is neither: the first run that annotates builds it at --eggnog_db.

    -

    The thresholds are deliberately wide, since AMAP is not restricted to tuberculosis. A 4.4 Mb MTBC genome at 65% GC sits comfortably inside them, and so does most of the bacterial world.

    -

    Assembly quality#

    +

    WarningAvoid dots in sample IDs

    The classification cuts names at the first dot, so for G26689.1 it writes its tables as L4_G26689 while the pipeline waits for L4_G26689.1. The step fails, is tried twice more and is set aside, and the run finishes without that sample. Validation does not check this: stick to letters, digits and underscores.

    +

    Pre-flight validation

    #
    +

    VALIDATE_INPUTS runs before any computation, on every run except -profile test, which has no sample sheet. To run only the checks and stop:

    +
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --validate_only
    +

    A check that passes writes its results to results/pipeline_info/validation.log; one that fails stops the run and prints them. Two mistakes in the sheet itself stop it even earlier, before any task starts, with a message of the pipeline's own and no log at all: the drawing of the sheet above shows both.

    +

    Genome sizelog scale

    M. tuberculosis, M. bovis: 4.35ย toย 4.41ย MbE. coli: 4.64ย toย 5.59ย Mb
    100ย kb500ย kb10ย Mb20ย Mb
    • M. tuberculosis, M. bovis: 4.35ย toย 4.41ย Mb
    • E. coli: 4.64ย toย 5.59ย Mb

    GC content% of A, C, G and T

    tuberculosis complex: 65.6%E. coli: 50.5ย toย 50.8%
    0%20%25%50%75%80%100%
    • tuberculosis complex: 65.6%
    • E. coli: 50.5ย toย 50.8%

    N content% of the bases

    all six: under 0.01%
    0%1%5%10%15%
    • all six: under 0.01%

    Sequencescontigs, log scale

    closed genomes: 1Sakai: 3CDC1551 in 40 contigs
    1101005002,000
    • closed genomes: 1
    • Sakai: 3
    • CDC1551 in 40 contigs
    passeswarnswarns: highly fragmentedfails the run
    Where a genome has to sit, with the limits read off bin/validate_inputs.py, and where six of the seven genomes of the accuracy benchmark sit: inside every range, with no warning. The seventh, CP110619, is another assembly of H37Rv and is not measured here.
    +

    What fails the run

    #
    +
    CheckThreshold
    Genome sizebelow 100ย kb, or above 20ย Mb
    N contentabove 10%
    GC contentbelow 20% or above 80%
    Empty sequencesany record with no bases
    Duplicate contig headersany repeated identifier
    Sample sheetmissing sampleId or fasta column, no data rows, duplicate or empty sample IDs
    FASTA pathfile does not exist or cannot be parsed
    Bakta databasenot found, or no version.json or bakta.db in it
    eggNOG databaseeggnog_db.ok in it while the database is incomplete, since Nextflow then never builds it again
    Reference files--ref_annotation, --ref_genome or a reference argument that the run reads but that is neither on the host nor inside the container
    +

    What only warns

    #
    +
    CheckThreshold
    Genome sizebelow 500ย kb, or above 10ย Mb
    N contentabove 1%
    GC contentbelow 25% or above 75%
    Assembly fragmentationmore than 100 contigs, and again above 500
    Short contigsany contig under 200ย bp in a multi-contig assembly
    Non-standard charactersanything outside ATGCNRYSWKMBDHV
    eggNOG databasea bacteria.dmnd under 100ย MB, which EGGNOG_DB takes for a truncated download and builds again
    Mycobrowser table--mycobrowser_ref neither on the host nor inside the container, so the master table goes without Mycobrowser categories
    +

    Warnings do not stop a run. Add --strict_validation to have them stop it too.

    +

    An eggNOG database that does not exist yet is neither: the first run that annotates builds it at --eggnog_db.

    +

    The thresholds are deliberately wide, since AMAP is not restricted to tuberculosis. A 4.4ย Mb genome of the Mycobacterium tuberculosis complex (MTBC) at 65% GC sits comfortably inside them, and so does most of the bacterial world.

    +

    Assembly quality

    #

    The correction compares your annotation against a reference proteome, so assembly quality propagates directly into the result.

    -

    Contiguity matters. A gene broken across two contigs cannot be recovered. The more fragmented the assembly, the more reference genes turn up as missing rather than corrected.

    +

    Contiguity matters. A gene that a contig end cuts comes back in pieces, each under the gene's name and marked partial at its open end, the way NCBI marks it. A piece under 8ย residues is too short to place: in CDC1551 cut into 40 contigs, 22 of the 25 smaller pieces were placed, and the other three were five or six codons long. The more fragmented the assembly, the more partial genes; pieces of cut genes has the rule.

    Ns break coding sequences. A run of Ns inside a gene stops it translating, so it is dropped or flagged as a pseudogene. The 1% warning is worth taking seriously.

    -

    Contig names are kept

    AMAP runs Bakta with --keep-contig-headers, so the sequence names you supplied are the ones in BAKTA/ and in the corrected annotation, and the corrected tables say which contig each gene is on. Without that option Bakta relabels everything contig_1, contig_2 and so on, and classify_cases.py stops with an error on a multi-contig assembly rather than compare genes across contigs.

    -

    Closed genomes are read as circular from their header

    Bakta takes a sequence for a complete, circular replicon when the description after its name says so: complete, as NCBI's "complete genome" and "complete sequence" do, or circular=true, as Unicycler's does. It then joins the two ends, and a gene across the origin comes out whole. Lineage typing renames a genome of one sequence after its sample and keeps that description. A closed genome whose header says neither is annotated as linear, and a gene across its origin is cut in two there; add circular=true after its name to have it read as circular.

    -

    Draft assemblies

    Every contig of a draft assembly is numbered from 1, so the same coordinates turn up on several contigs. The correction keys every gene on its contig as well as its start, end and strand, only ever compares genes on the same contig, and translates each one from its own contig. Cut into 40 contigs, CDC1551 gets every gene that lies within one contig exactly as the closed genome does.

    -

    Reference data#

    +

    NoteContig names are kept

    AMAP runs Bakta with --keep-contig-headers, so the sequence names you supplied are the ones in BAKTA/ and in the corrected annotation, and the corrected tables say which contig each gene is on. Without that option Bakta relabels everything contig_1, contig_2 and so on, and classify_cases.py stops with an error on a multi-contig assembly rather than compare genes across contigs.

    +

    NoteClosed genomes are read as circular from their header

    Bakta takes a sequence for a complete, circular replicon when the description after its name says so: complete, as NCBI's "complete genome" and "complete sequence" do, or circular=true, as Unicycler's does. It then joins the two ends, and a gene across the origin comes out whole. Lineage typing renames a genome of one sequence after its sample and keeps that description. A closed genome whose header says neither is annotated as linear, and a gene across its origin is cut in two there; add circular=true after its name to have it read as circular.

    +

    NoteDraft assemblies

    Every contig of a draft assembly is numbered from 1, so the same coordinates turn up on several contigs. The correction keys every gene on its contig as well as its start, end and strand, only ever compares genes on the same contig, and translates each one from its own contig. Cut into 40 contigs, CDC1551 gets every gene that lies within one contig exactly as the closed genome does.

    +

    Reference data

    #

    For MTBC you need nothing else. The container bundles the H37Rv proteome, the Bakta light database, the Mycobrowser table and the pathotypr lineage markers. See containers for the full list.

    -

    For any other organism, supply your own reference with --ref_annotation and --ref_genome. See custom reference.

    -

    Next steps#

    +

    For any other organism, supply your own reference with --ref_annotation and --ref_genome. See custom reference.

    +

    Next steps

    #
    -

    Edit this page on GitHub

    - +

    Edit this page on GitHub

    +
    -
    - diff --git a/docs/getting-started/inputs.md b/docs/getting-started/inputs.md index 93d7256..c4f356c 100644 --- a/docs/getting-started/inputs.md +++ b/docs/getting-started/inputs.md @@ -11,7 +11,7 @@ A tab-separated file with a header row and exactly two columns: sampleId fasta G26689 /data/genomes/G26689.fasta G26690 /data/genomes/G26690.fasta -L4_G12345 ./assemblies/L4_G12345.fa +G12345 ./assemblies/G12345.fa ``` @@ -28,21 +28,27 @@ common reason a sample sheet is rejected. > [!WARNING] > **Avoid dots in sample IDs** > -> Several scripts derive the sample name by cutting at the first dot, so -> `G26689.1` becomes `G26689` and two such samples would collide. Stick to -> letters, digits and underscores. +> The classification cuts names at the first dot, so for `G26689.1` it writes +> its tables as `L4_G26689` while the pipeline waits for `L4_G26689.1`. The +> step fails, is tried twice more and is set aside, and the run finishes +> without that sample. Validation does not check this: stick to letters, +> digits and underscores. ## Pre-flight validation -`VALIDATE_INPUTS` runs before any computation, on every run. To run only the -checks and stop: +`VALIDATE_INPUTS` runs before any computation, on every run except +`-profile test`, which has no sample sheet. To run only the checks and stop: ```bash nextflow run PathoGenOmics-Lab/AMAP -profile docker \ --tsv samples.tsv --validate_only ``` -Results are written to `results/pipeline_info/validation.log`. +A check that passes writes its results to +`results/pipeline_info/validation.log`; one that fails stops the run and prints +them. Two mistakes in the sheet itself stop it even earlier, before any task +starts, with a message of the pipeline's own and no log at all: the drawing of +the sheet above shows both. @@ -80,7 +86,7 @@ An eggNOG database that does not exist yet is neither: the first run that annotates builds it at `--eggnog_db`. The thresholds are deliberately wide, since AMAP is not restricted to -tuberculosis. A 4.4 Mb MTBC genome at 65% GC sits comfortably inside them, and +tuberculosis. A 4.4 Mb genome of the *Mycobacterium tuberculosis* complex (MTBC) at 65% GC sits comfortably inside them, and so does most of the bacterial world. ## Assembly quality @@ -88,9 +94,13 @@ so does most of the bacterial world. The correction compares your annotation against a reference proteome, so assembly quality propagates directly into the result. -**Contiguity matters.** A gene broken across two contigs cannot be recovered. -The more fragmented the assembly, the more reference genes turn up as missing -rather than corrected. +**Contiguity matters.** A gene that a contig end cuts comes back in pieces, +each under the gene's name and marked partial at its open end, the way NCBI +marks it. A piece under 8 residues is too short to place: in CDC1551 cut into +40 contigs, 22 of the 25 smaller pieces were placed, and the other three were +five or six codons long. The more fragmented the assembly, the more partial +genes; [pieces of cut genes](../how-it-works/correction-algorithm.md#pieces-of-cut-genes) +has the rule. **Ns break coding sequences.** A run of Ns inside a gene stops it translating, so it is dropped or flagged as a pseudogene. The 1% warning is worth taking diff --git a/docs/getting-started/installation.html b/docs/getting-started/installation.html index fb12cd6..758e50f 100644 --- a/docs/getting-started/installation.html +++ b/docs/getting-started/installation.html @@ -4,138 +4,161 @@ Installation | AMAP docs - + + - + - + - + - +
    - AMAP + AMAP
    - + + +
    -

    Docs / Getting started

    +

    Docs / Getting started

    Describes AMAP on main, ahead of 1.0.1: what changed

    Installation

    -

    AMAP needs two things on your machine: Nextflow, which runs the pipeline, and a container engine, which supplies every tool the pipeline calls. You do not install Bakta, Miniprot, eggNOG-mapper or any of the rest yourself; they all live inside the AMAP image.

    +

    Nextflow, a container engine, and running without one.

    +

    3 min read

    +
    On this page
    + +

    AMAP needs two things on your machine: Nextflow, which runs the pipeline, and a container engine, which supplies every tool the pipeline calls. You do not install Bakta, miniprot, eggNOG-mapper or any of the rest yourself; they all live inside the AMAP image.

    You also do not need to clone this repository. Nextflow pulls the pipeline straight from GitHub.

    -

    What you install

    • Nextflow 24.04 or newer, on Java 17 or newer
    • Docker, or Singularity or Apptainer on a cluster

    What the first run builds

    • The eggNOG database, in the folder --eggnog_db names, kept for every run after

    paururo/amaplinux/amd64

    Tools

    Bakta 1.12.0miniprot 0.18Diamond 2.1.24eggNOG-mapper 2.1.13pangene 1.1pathotypr 1.0.0gfatools

    Runtimes

    Python 3.9R 4.4

    Data

    Bakta database light/reference/ 9 files: the H37Rv genome, proteome and GenBank, Bakta's false positives on it, Mycobrowser's table and the lineage markers
    Two things to install; everything AMAP runs comes in one image, its versions read off docker/Dockerfile. Only the eggNOG database stays outside: too large to ship, it is built on the first run that needs it.
    -

    1. Nextflow#

    -

    Nextflow needs Java 17 or newer, and Conda brings both:

    -
    conda create -n nextflow-env
-conda activate nextflow-env
-conda install -c bioconda nextflow
    -

    If you do not have Conda, Miniconda is a lightweight installer. Any other Nextflow installation works equally well, see the Nextflow documentation.

    -

    AMAP requires Nextflow 24.04 or newer, which is enforced by the pipeline manifest.

    -

    2. A container engine#

    +

    What you install

    • Nextflow 24.04 or newer, on Javaย 17 or newer
    • Docker, or Singularity or Apptainer on a cluster

    What the first run builds

    • The eggNOG database, in the folder --eggnog_db names, kept for every run after

    paururo/amaplinux/amd64

    Tools

    Bakta 1.12.0miniprot 0.18Diamond 2.1.24eggNOG-mapper 2.1.13pangene 1.1pathotypr 1.0.0gfatools

    Runtimes

    Python 3.9R 4.4

    Data

    Bakta database light/reference/ 9 files: the H37Rv genome, proteome and GenBank, Bakta's false positives on it, Mycobrowser's table and the lineage markers
    Two things to install; everything AMAP runs comes in one image, its versions read off docker/Dockerfile. Only the eggNOG database stays outside: too large to ship, it is built on the first run that needs it.
    +

    1. Nextflow

    #
    +

    Nextflow needs Javaย 17 or newer, and Conda brings both:

    +
    Terminal
    $ conda create -n nextflow-env -c conda-forge -c bioconda nextflow
+$ conda activate nextflow-env
    +

    If you do not have Conda, Miniforge is the one to install: it comes set up for conda-forge, the channel Bioconda builds on. Any other Nextflow installation works equally well, see the Nextflow documentation.

    +

    AMAP requires Nextflowย 24.04 or newer, which is enforced by the pipeline manifest.

    +

    2. A container engine

    #

    Pick whichever your machine allows.

    Docker

    Best on a laptop or workstation where you have root. Follow the Docker installation guide for your platform.

    Singularity / Apptainer

    The usual choice on an HPC cluster, where Docker is normally not permitted. Follow the Apptainer installation guide. Many clusters already provide it as a module:

    -
    module load singularity   # or: module load apptainer
    -

    3. Check the installation#

    -
    nextflow -version
-docker --version        # or: singularity --version
    -

    4. Run the test profile#

    -

    Before pointing AMAP at real data, confirm the whole chain works. The test profile annotates the reference genome bundled in the container, so it needs no data of your own and nothing to download beyond the image:

    -
    nextflow run PathoGenOmics-Lab/AMAP -profile docker,test
    -

    The first run downloads the AMAP image, which is several gigabytes because it bundles the Bakta database and the H37Rv reference files. Expect that to take a while. Later runs reuse the cached image.

    -

    Check inputs before a long run

    Once you have your own samples, --validate_only checks every FASTA and database path without executing anything:

    -
    nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --validate_only
    +
    Terminal
    $ module load singularity   # or: module load apptainer
    +

    3. Check the installation

    #
    +
    Terminal
    $ nextflow -version
+$ docker info --format 'Docker can give containers {{.NCPU}} CPUs and {{.MemTotal}} bytes'
    +

    On a machine with Docker Desktop running, the second line prints something like this:

    +
    Output
    Docker can give containers 14 CPUs and 8217014272 bytes
    +

    Unlike docker --version, it fails when the Docker daemon is not running, and it prints the memory your containers can really have. Under 8ย GB? Raise it in Docker Desktop under Settings, Resources, Memory. With Singularity or Apptainer, singularity --version is the check.

    +

    4. Run the test profile

    #
    +

    Before pointing AMAP at real data, confirm the whole chain works. The test profile annotates the H37Rv genome bundled in the image, so it needs no data of your own, and --skip_functional_annotation keeps it to the correction:

    +
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile docker,test \
+    --skip_functional_annotation --outdir amap-test
    +

    It downloads nothing beyond the pipeline and its image, writes amap-test/ and Nextflow's work/ in the folder you run it from, and ends with a โœ“ COMPLETED block. On an Apple M4 Pro, under emulation and with the image already pulled, it took 2ย min 47ย s.

    +

    The first run downloads the AMAP image, about 3.4ย GB compressed and 12ย GB on disk, because it bundles the Bakta database and the H37Rv reference files. Expect that to take a while. Later runs reuse the cached image.

    +

    The eggNOG database

    #
    +

    Without --skip_functional_annotation, the test also builds the eggNOG database, the one thing the image does not hold. It is the largest thing AMAP keeps: eggnog.db alone unpacks to 41.4ย GB from a 6.8ย GB download, and the build also downloads the 9.5ย GB eggNOG proteome, makes the bacteria-only Diamond index from it and deletes it. It is built once, in the folder --eggnog_db names, and every run after reuses it, so point that somewhere with room. The annotation then loads it into memory and asks for 50ย GB, where the test profile caps every task at 8ย GB and 1ย h, so keep --skip_functional_annotation for the test.

    +

    TipCheck inputs before a long run

    Once you have your own samples, --validate_only checks every FASTA and database path without executing anything:

    +
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --validate_only

    See inputs for what it looks for.

    -

    Running without a container#

    -

    Possible but not recommended: you take on installing Bakta, Miniprot, eggNOG-mapper, Diamond, pangene, pathotypr, R and the report packages yourself, and pointing AMAP at each reference file with the reference arguments.

    -

    The repository ships an environment.yml that reproduces the software stack under Conda:

    -
    conda env create -f environment.yml
-conda activate amap
    +

    Running without aย container

    #
    +

    Possible but not recommended: you take on installing Bakta, miniprot, eggNOG-mapper, Diamond, pangene, pathotypr, R and the report packages yourself, and pointing AMAP at each reference file with the reference arguments.

    +

    The repository ships an environment.yml that reproduces the software stack under Conda:

    +
    Terminal
    $ conda env create -f environment.yml
+$ conda activate amap

    You will still need to supply the Bakta database and the reference files, which the container would otherwise provide. See containers for what the image contains.

    -

    Next steps#

    +

    Next steps

    #
    -

    Edit this page on GitHub

    - +

    Back to the top

    +

    Edit this page on GitHub

    +
    -
    - diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index cb0e4cc..507d16e 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -2,7 +2,7 @@ AMAP needs two things on your machine: **Nextflow**, which runs the pipeline, and a **container engine**, which supplies every tool the pipeline calls. You do -not install Bakta, Miniprot, eggNOG-mapper or any of the rest yourself; they all +not install Bakta, miniprot, eggNOG-mapper or any of the rest yourself; they all live inside the AMAP image. You also do not need to clone this repository. Nextflow pulls the pipeline @@ -15,14 +15,14 @@ straight from GitHub. Nextflow needs Java 17 or newer, and Conda brings both: ```bash -conda create -n nextflow-env +conda create -n nextflow-env -c conda-forge -c bioconda nextflow conda activate nextflow-env -conda install -c bioconda nextflow ``` -If you do not have Conda, [Miniconda](https://docs.conda.io/en/latest/miniconda.html) -is a lightweight installer. Any other Nextflow installation works equally well, -see the [Nextflow documentation](https://www.nextflow.io/docs/latest/getstarted.html). +If you do not have Conda, [Miniforge](https://github.com/conda-forge/miniforge) +is the one to install: it comes set up for conda-forge, the channel Bioconda +builds on. Any other Nextflow installation works equally well, see the +[Nextflow documentation](https://www.nextflow.io/docs/latest/getstarted.html). AMAP requires Nextflow 24.04 or newer, which is enforced by the pipeline manifest. @@ -52,22 +52,52 @@ module load singularity # or: module load apptainer ```bash nextflow -version -docker --version # or: singularity --version +docker info --format 'Docker can give containers {{.NCPU}} CPUs and {{.MemTotal}} bytes' ``` +On a machine with Docker Desktop running, the second line prints something like +this: + +```text output +Docker can give containers 14 CPUs and 8217014272 bytes +``` + +Unlike `docker --version`, it fails when the Docker daemon is not running, and +it prints the memory your containers can really have. Under 8 GB? Raise it in +Docker Desktop under Settings, Resources, Memory. With Singularity or Apptainer, +`singularity --version` is the check. + ## 4. Run the test profile Before pointing AMAP at real data, confirm the whole chain works. The test -profile annotates the reference genome bundled in the container, so it needs -no data of your own and nothing to download beyond the image: +profile annotates the H37Rv genome bundled in the image, so it needs no data of +your own, and `--skip_functional_annotation` keeps it to the correction: ```bash -nextflow run PathoGenOmics-Lab/AMAP -profile docker,test +nextflow run PathoGenOmics-Lab/AMAP -profile docker,test \ + --skip_functional_annotation --outdir amap-test ``` -The first run downloads the AMAP image, which is several gigabytes because it -bundles the Bakta database and the H37Rv reference files. Expect that to take a -while. Later runs reuse the cached image. +It downloads nothing beyond the pipeline and its image, writes `amap-test/` and +Nextflow's `work/` in the folder you run it from, and ends with a `โœ“ COMPLETED` +block. On an Apple M4 Pro, under emulation and with the image already pulled, +it took 2 min 47 s. + +The first run downloads the AMAP image, about 3.4 GB compressed and 12 GB on +disk, because it bundles the Bakta database and the H37Rv reference files. +Expect that to take a while. Later runs reuse the cached image. + +### The eggNOG database + +Without `--skip_functional_annotation`, the test also builds the eggNOG +database, the one thing the image does not hold. It is the largest thing AMAP +keeps: `eggnog.db` alone unpacks to 41.4 GB from a 6.8 GB download, and the +build also downloads the 9.5 GB eggNOG proteome, makes the bacteria-only Diamond +index from it and deletes it. It is built once, in the folder `--eggnog_db` +names, and every run after reuses it, so point that somewhere with room. The +annotation then loads it into memory and asks for 50 GB, where the test profile +caps every task at 8 GB and 1 h, so keep `--skip_functional_annotation` for the +test. > [!TIP] > **Check inputs before a long run** @@ -84,7 +114,7 @@ while. Later runs reuse the cached image. ## Running without a container -Possible but not recommended: you take on installing Bakta, Miniprot, +Possible but not recommended: you take on installing Bakta, miniprot, eggNOG-mapper, Diamond, pangene, pathotypr, R and the report packages yourself, and pointing AMAP at each reference file with the [reference arguments](../user-guide/parameters.md#reference-and-database-arguments). diff --git a/docs/getting-started/quickstart.html b/docs/getting-started/quickstart.html index d39b609..2fc7dd3 100644 --- a/docs/getting-started/quickstart.html +++ b/docs/getting-started/quickstart.html @@ -4,174 +4,211 @@ Quickstart | AMAP docs - + + - + - + - + - +
    - AMAP + AMAP
    - + + +
    -

    Docs / Getting started

    +

    Docs / Getting started

    Describes AMAP on main, ahead of 1.0.1: what changed

    Quickstart

    +

    From an installed Nextflow to an open report: the test, a sample sheet, the check, the run and the results.

    +

    3 min read

    +
    On this page
    +

    This walks from an installed Nextflow to an open report. If Nextflow or a container engine is still missing, start at installation.

    -

    1. Write a sample sheet#

    -

    AMAP reads a tab-separated file with two columns, sampleId and fasta:

    -
    samples.tsv
    sampleId	fasta
-G26689	/data/genomes/G26689.fasta
-G26690	/data/genomes/G26690.fasta
    +

    1. Check the plumbing

    #
    +

    The test profile annotates the H37Rv genome that ships inside the image, so it needs no data of your own, and --skip_functional_annotation keeps it to the correction, with no eggNOG database to build:

    +
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile docker,test \
+    --skip_functional_annotation --outdir amap-test
    +

    It writes amap-test/ and work/ where you run it, and ends with a โœ“ COMPLETED block. On an Apple M4 Pro, under emulation and with the image already pulled, it took 2ย min 47ย s; the first run also pulls the image, about 3.4ย GB compressed and 12ย GB on disk, once.

    +

    2. Write aย sample sheet

    #
    +

    AMAP reads a tab-separated file with two columns, sampleId and fasta:

    +
    samples.tsv
    sampleId	fasta
+G26689	/data/genomes/G26689.fasta
+G26690	/data/genomes/G26690.fasta

    The separator must be a real tab. Paths can be absolute, or relative to where you launch Nextflow. See preparing inputs for the details and the checks AMAP runs over them.

    -

    2. Check the inputs#

    +

    3. Check the inputs

    #

    Cheap, and it catches the mistakes that would otherwise surface twenty minutes into a run:

    -
    nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --validate_only
    -

    3. Run it#

    +
    Terminal
    $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --validate_only
    +

    When the check passes, its verdict and any warnings are in results/pipeline_info/validation.log, and the terminal says only that the step ran. When it fails, the run stops and the terminal prints the whole report.

    +

    4. Run it

    #

    Pick what you have, and copy the command:

    -
    +
    Where will it run?
    -
    - - - +
    + + +
    Which genomes?
    -
    - - +
    + +
    Anything else?
    - +
    -
    - 
    
-    
      
-    
+  
      
+    
      Check it first: the same command, with --validate_only
      
+    
      $ nextflow run PathoGenOmics-Lab/AMAP \
+    -profile docker \
+    --tsv samples.tsv \
+    --validate_only
      
+    
      Seconds per genome. A check that fails stops and prints its whole report. One that passes prints nothing, warnings included: its verdict is in results/pipeline_info/validation.log, here for the tutorial's genome:
      
+    
      Results: 1 OK, 0 warnings, 0 failed
+Total: 0 error(s), 0 warning(s)
      
+    
      Then run it
      
+    
      $ nextflow run PathoGenOmics-Lab/AMAP \
+    -profile docker \
+    --tsv samples.tsv
      
+    
      • -profileEvery tool runs inside the AMAP image, through Docker.
      • --tsvYour sample sheet: sampleId and fasta, tab separated.
      
+    
   
      
-
      The command for your machine and your genomes, with what each option does. The same as on the front page.
      
+
      The command for your machine and your genomes, with what each option does. The same as on the front page.
      
 
      Or start from one of these:
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
      
-
      Without a scheduler profile every task is capped at the machine's CPUs and 16 GB, and with the scheduler profiles at 48 CPUs and 128 GB, whichever order the profiles come in; a larger request, such as the 50 GB the eggNOG annotation asks for, is lowered to the cap. Nextflow refuses to start a task asking for more memory than the machine has, so on a smaller machine, lower the ceiling:
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --max_cpus 4 --max_memory 12.GB
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
      
+
      Without an executor profile every task is capped at the machine's CPUs and 16ย GB, and with the executor profiles at 48ย CPUs and 128ย GB, whichever order the profiles come in; a larger request is lowered to the cap. Nextflow refuses to start a task asking for more memory than the machine has, so on a smaller machine, lower the ceiling to what docker info says your containers can have, and leave eggNOG out: it loads its whole database into memory, about 45ย GB, whatever the ceiling.
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --max_cpus 4 --max_memory 8.GB \
+    --skip_functional_annotation
      
 
      The tutorial has the per-step figures and what to do when the machine cannot reach them.
      
-
      4. Read the results#
      
-
      Everything lands under results/ unless you set --outdir. The three things to open first:
      
-
      WhatWhere
      The full interactive reportresults/final_report.html
      Corrected annotation, per sampleresults/AMAP/{sampleId}/*-AMAP.gff3
      Protein sequences for downstream toolsresults/AMAP/{sampleId}/*-AMAP.clean.faa
      
-
      Which protein file do I want?
      *-AMAP.faa holds every gene, pseudogenes included, so it mirrors the annotation exactly. *-AMAP.clean.faa drops pseudogenes and sequences with internal stop codons, which is what orthology, phylogenomics and dN/dS tools expect. Reach for the clean one unless you specifically need the complete set.
      
+
      5. Read the results
      #
      
+
      Everything lands under results/ unless you set --outdir. Each sample's folder is named {lineage}_{sampleId}: lineage typing puts the lineage in front, so G26689 comes out as L4_G26689, and under --skip_typing it is the sample ID alone (outputs has the details). The things to open first:
      
+
      WhatWhere
      Each genome and each gene, question by question, in every runresults/amap_report.html
      The full report, after a run with eggNOGresults/final_report.html
      Corrected annotation, per sample, after a run with eggNOGresults/AMAP/{lineage}_{sampleId}/*-AMAP.gff3
      Protein sequences for downstream tools, after a run with eggNOGresults/AMAP/{lineage}_{sampleId}/*-AMAP.clean.faa
      The corrected genes, their proteins and sequences, in every runresults/AMAP/{lineage}_{sampleId}/*.AMAP.tsv, .faa, .fna
      
+
      NoteWhich protein file do I want?
      *-AMAP.faa holds every gene, pseudogenes included, so it mirrors the annotation exactly. *-AMAP.clean.faa drops pseudogenes and sequences with internal stop codons, which is what orthology, phylogenomics and dN/dS tools expect. Reach for the clean one unless you specifically need the complete set.
      
 
      The outputs page covers every directory.
      
-
      5. Resume an interrupted run#
      
-
      Nextflow caches completed tasks. If a run dies, restart it with -resume and it picks up where it stopped:
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv -resume
      
+
      6. Resume anย interrupted run
      #
      
+
      Nextflow caches completed tasks. If a run dies, restart it with -resume and it picks up where it stopped:
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv -resume
      
 
      Keep the same parameters. Changing an input invalidates the cache for every task downstream of it, which is what you want, but it does mean the run redoes that work.
      
-
      Common variations#
      
-
      A non-MTBC organism. Lineage typing and Mycobrowser are specific to tuberculosis, so turn them off and supply your own reference:
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv \
-    --skip_typing --skip_mycobrowser \
-    --genus Escherichia --species "Escherichia coli" \
-    --ref_annotation my_reference.gbk --ref_genome my_reference.fasta
      
+
      Common variations
      #
      
+
      An organism outside the Mycobacterium tuberculosis complex (MTBC). Lineage typing and Mycobrowser are specific to the complex, so turn them off and supply your own reference:
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv \
+    --skip_typing --skip_mycobrowser \
+    --genus Escherichia --species "Escherichia coli" \
+    --ref_annotation my_reference.gbk --ref_genome my_reference.fasta
      
 
      See custom reference.
      
-
      Just the corrected genes. If you do not need functional annotation, duplications, the pangenome or the report, stop after the correction step. It is much faster, and leaves the corrected genes as a table with their proteins and sequences; the GFF3 and GenBank files come after eggNOG, so there are none:
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --skip_functional_annotation
      
-
      Next steps#
      
-
-
      Edit this page on GitHub
      
-
+
      Just the corrected genes. If you do not need functional annotation, duplications or the pangenome, stop after the correction step. It is much faster, and leaves the corrected genes as a table with their proteins and sequences; the GFF3 and GenBank files come after eggNOG, so there are none. There is no final_report.html; amap_report.html is still written, without its eggNOG sections:
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --skip_functional_annotation
      
+
      Next steps
      #
      
+
+
      Back to the top
      
+
      Edit this page on GitHub
      
+
 
      
-
 
 
-
 
 
 
diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md
index 8fa4094..c469f40 100644
--- a/docs/getting-started/quickstart.md
+++ b/docs/getting-started/quickstart.md
@@ -3,7 +3,23 @@
 This walks from an installed Nextflow to an open report. If Nextflow or a
 container engine is still missing, start at [installation](installation.md).
 
-## 1. Write a sample sheet
+## 1. Check the plumbing
+
+The test profile annotates the H37Rv genome that ships inside the image, so it
+needs no data of your own, and `--skip_functional_annotation` keeps it to the
+correction, with no eggNOG database to build:
+
+```bash
+nextflow run PathoGenOmics-Lab/AMAP -profile docker,test \
+    --skip_functional_annotation --outdir amap-test
+```
+
+It writes `amap-test/` and `work/` where you run it, and ends with a
+`โœ“ COMPLETED` block. On an Apple M4 Pro, under emulation and with the image
+already pulled, it took 2 min 47 s; the first run also pulls the image, about
+3.4 GB compressed and 12 GB on disk, once.
+
+## 2. Write a sample sheet
 
 AMAP reads a tab-separated file with two columns, `sampleId` and `fasta`:
 
@@ -17,7 +33,7 @@ The separator must be a real tab. Paths can be absolute, or relative to where
 you launch Nextflow. See [preparing inputs](inputs.md) for the details and the
 checks AMAP runs over them.
 
-## 2. Check the inputs
+## 3. Check the inputs
 
 Cheap, and it catches the mistakes that would otherwise surface twenty minutes
 into a run:
@@ -27,7 +43,11 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
     --tsv samples.tsv --validate_only
 ```
 
-## 3. Run it
+When the check passes, its verdict and any warnings are in
+`results/pipeline_info/validation.log`, and the terminal says only that the
+step ran. When it fails, the run stops and the terminal prints the whole report.
+
+## 4. Run it
 
 Pick what you have, and copy the command:
 
@@ -54,30 +74,38 @@ nextflow run PathoGenOmics-Lab/AMAP \
     -profile singularity,slurm --tsv samples.tsv
 ```
 
-Without a scheduler profile every task is capped at the machine's CPUs and
-16 GB, and with the scheduler profiles at 48 CPUs and 128 GB, whichever order
-the profiles come in; a larger request, such as the 50 GB the eggNOG annotation
-asks for, is lowered to the cap. Nextflow refuses to start a task asking for
-more memory than the machine has, so on a smaller machine, lower the ceiling:
+Without an executor profile every task is capped at the machine's CPUs and
+16 GB, and with the executor profiles at 48 CPUs and 128 GB, whichever order
+the profiles come in; a larger request is lowered to the cap. Nextflow refuses
+to start a task asking for more memory than the machine has, so on a smaller
+machine, lower the ceiling to what `docker info` says your containers can have,
+and leave eggNOG out: it loads its whole database into memory, about 45 GB,
+whatever the ceiling.
 
 ```bash
 nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --max_cpus 4 --max_memory 12.GB
+    --tsv samples.tsv --max_cpus 4 --max_memory 8.GB \
+    --skip_functional_annotation
 ```
 
 The [tutorial](tutorial.md#the-machine) has the per-step figures and what to do
 when the machine cannot reach them.
 
-## 4. Read the results
+## 5. Read the results
 
-Everything lands under `results/` unless you set `--outdir`. The three things to
-open first:
+Everything lands under `results/` unless you set `--outdir`. Each sample's
+folder is named `{lineage}_{sampleId}`: lineage typing puts the lineage in
+front, so `G26689` comes out as `L4_G26689`, and under `--skip_typing` it is the
+sample ID alone ([outputs](../user-guide/outputs.md) has the details). The
+things to open first:
 
 | What | Where |
 | :--- | :--- |
-| The full interactive report | `results/final_report.html` |
-| Corrected annotation, per sample | `results/AMAP/{sampleId}/*-AMAP.gff3` |
-| Protein sequences for downstream tools | `results/AMAP/{sampleId}/*-AMAP.clean.faa` |
+| Each genome and each gene, question by question, in every run | `results/amap_report.html` |
+| The full report, after a run with eggNOG | `results/final_report.html` |
+| Corrected annotation, per sample, after a run with eggNOG | `results/AMAP/{lineage}_{sampleId}/*-AMAP.gff3` |
+| Protein sequences for downstream tools, after a run with eggNOG | `results/AMAP/{lineage}_{sampleId}/*-AMAP.clean.faa` |
+| The corrected genes, their proteins and sequences, in every run | `results/AMAP/{lineage}_{sampleId}/*.AMAP.tsv`, `.faa`, `.fna` |
 
 > [!NOTE]
 > **Which protein file do I want?**
@@ -90,7 +118,7 @@ open first:
 
 The [outputs page](../user-guide/outputs.md) covers every directory.
 
-## 5. Resume an interrupted run
+## 6. Resume an interrupted run
 
 Nextflow caches completed tasks. If a run dies, restart it with `-resume` and it
 picks up where it stopped:
@@ -105,8 +133,9 @@ work.
 
 ## Common variations
 
-**A non-MTBC organism.** Lineage typing and Mycobrowser are specific to
-tuberculosis, so turn them off and supply your own reference:
+**An organism outside the *Mycobacterium tuberculosis* complex (MTBC).**
+Lineage typing and Mycobrowser are specific to the complex, so turn them off
+and supply your own reference:
 
 ```bash
 nextflow run PathoGenOmics-Lab/AMAP -profile docker \
@@ -119,9 +148,11 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
 See [custom reference](../user-guide/custom-reference.md).
 
 **Just the corrected genes.** If you do not need functional annotation,
-duplications, the pangenome or the report, stop after the correction step. It is
-much faster, and leaves the corrected genes as a table with their proteins and
-sequences; the GFF3 and GenBank files come after eggNOG, so there are none:
+duplications or the pangenome, stop after the correction step. It is much
+faster, and leaves the corrected genes as a table with their proteins and
+sequences; the GFF3 and GenBank files come after eggNOG, so there are none. There
+is no `final_report.html`; `amap_report.html` is still written, without its
+eggNOG sections:
 
 ```bash
 nextflow run PathoGenOmics-Lab/AMAP -profile docker \
@@ -132,4 +163,4 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
 
 - [Running AMAP](../user-guide/running.md): profiles, schedulers and larger runs
 - [Parameters](../user-guide/parameters.md): the full reference
-- [How it works](../how-it-works/index.md): what the correction actually does
+- [Overview](../how-it-works/index.md): what the correction actually does
diff --git a/docs/getting-started/tutorial.html b/docs/getting-started/tutorial.html
index e607583..20d7e08 100644
--- a/docs/getting-started/tutorial.html
+++ b/docs/getting-started/tutorial.html
@@ -4,263 +4,313 @@
 
 
 Tutorial | AMAP docs
-
+
 
+
 
 
 
-
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
      
   
      
     
-      AMAP
+      AMAP
     
     
   
      
 
      
 
      
-
+  
+  
+
      
 
      
-
      Docs / Getting started
      
-
      Tutorial: from an empty folder to an annotated genome
      
-
      Every command spelled out, nothing assumed. It uses a real genome you can download in one line, so you can follow along and compare your numbers against the ones printed here. Those numbers come from actual runs of these commands, not from memory, made on 18 October 2026 with AMAP at commit 947a99d. The runs skipped eggNOG, with --skip_functional_annotation. None of the numbers depends on it, and the one table shown here that is written after it was made by hand from the same run, as the sample check explains.
      
-
      The genome is Mycobacterium tuberculosis CDC1551, corrected against the H37Rv reference that ships inside the AMAP container. At the end there is a second pass showing the same thing for a non-MTBC organism.
      
-
      The correction itself takes minutes on one genome. What dominates the first run is setup: pulling a 12 GB image and building the eggNOG database, both of which happen once and are reused afterwards. Expect the first run to take considerably longer than every run after it, and do not read a long silence during the database build as a failure.
      
-
      1. What you need#
      
+
      Docs / Getting started
      Describes AMAP on main, ahead of 1.0.1: what changed
      
+
      Tutorial: from anย empty folder to anย annotated genome
      
+
      One real genome, start to finish, with every number.
      
+
      10 min read
      
+
      On this page
      
+
+
      Every command spelled out, nothing assumed. It uses a real genome you can download in one line, so you can follow along and compare your numbers against the ones printed here. They come from actual runs of these commands, made on 18 October 2026 with AMAP at commit 947a99d. For exactly these numbers, add this to each nextflow run:
      
+
      -r 947a99dee49d2a2429e014296ac35ba08e911108
      
+
      Pinning the pipeline says why the whole hash.
      
+
      The runs skipped eggNOG, with --skip_functional_annotation, and no number here depends on it. The one table eggNOG's step writes, in the sample check, was made by hand from the same run.
      
+
      The genome is Mycobacterium tuberculosis CDC1551, corrected against the H37Rv reference that ships inside the AMAP container. At the end there is a second pass showing the same thing for an organism outside the Mycobacterium tuberculosis complex (MTBC).
      
+
      The correction itself takes minutes on one genome: 2ย min 16ย s here, without eggNOG. What dominates the first full run is setup: pulling the image, 3.4ย GB compressed and 12ย GB on disk, and building the eggNOG database, both of which happen once and are reused afterwards. Expect the first run to take considerably longer than every run after it, and do not read a long silence during the database build as a failure.
      
+
      1. What you need
      #
      
 
      Nextflow and a container engine. Every tool the pipeline itself calls lives inside the AMAP image, so there is nothing else to install.
      
 
      The easiest way to get Nextflow is Conda. If you do not have it, Miniforge is the one to install:
      
-
      conda create -n nextflow-env -c conda-forge -c bioconda nextflow
-conda activate nextflow-env
      
-
      Nextflow needs Java 17 or newer. Check what you have:
      
-
      java -version
-nextflow -version
      
-
      Java 8 is not enough
      Nextflow will refuse to start on Java 8 with a message about an unsupported class file version. Install a newer JDK, for example conda install -c conda-forge openjdk=17.
      
-
      Then a container engine. Docker on a laptop, Singularity or Apptainer on a cluster:
      
-
      docker --version        # or: singularity --version
      
+
      Terminal
      $ conda create -n nextflow-env -c conda-forge -c bioconda nextflow
+$ conda activate nextflow-env
      
+
      Nextflow needs Javaย 17 or newer. Check what you have:
      
+
      Terminal
      $ java -version
+$ nextflow -version
      
+
      WarningNextflow needs Javaย 17 or newer
      On Javaย 8 or 11, Nextflow stops with "Cannot find Java or it's a wrong version". Install a newer JDK, for example conda install -c conda-forge openjdk=17.
      
+
      Then a container engine. Docker on a laptop, Singularity or Apptainer on a cluster. With Docker, ask it what your containers can have:
      
+
      Terminal
      $ docker info --format 'Docker can give containers {{.NCPU}} CPUs and {{.MemTotal}} bytes'
      
+
      Output
      Docker can give containers 14 CPUs and 8217014272 bytes
      
+
      That is the machine behind this page: an Apple M4 Pro, with Docker Desktop given 8ย GB of its 24. The command fails when the Docker daemon is not running, which docker --version does not catch. Under 8ย GB? Raise it in Docker Desktop under Settings, Resources, Memory. On a cluster, singularity --version.
      
 
      Full details, including running without a container, are on the installation page.
      
-
      On an Apple Silicon Mac, read this first
      The AMAP image is linux/amd64 only, so on an M-series Mac every step runs under emulation. The test profile in step 2 is fine. A real genome is slow enough that a working run can look like a hung one, which is the part worth knowing before you start wondering what you broke.
      
+
      WarningOn an Apple Silicon Mac, read this first
      The AMAP image is linux/amd64 only, so on an M-series Mac every step runs under emulation. The test profile in stepย 2 is fine. A real genome is slow enough that a working run can look like a hung one, which is the part worth knowing before you start wondering what you broke.
      
 
      It is a reasonable way to learn what the pipeline does and to check your sample sheet. For results you intend to use, move to an amd64 machine or a cluster.
      
-
      The machine#
      
+
      The machine
      #
      
 
      Worth checking before you start, because this is what stops a run rather than anything you will have got wrong.
      
-
      Disk goes first. The container is 12 GB unpacked. The eggNOG database is built on first use and cached: AMAP skips the 60 GB Diamond core that eggNOG-mapper would otherwise fetch and builds a bacteria-only index of about 3 GB instead, on top of the metadata it downloads. Nextflow keeps every intermediate under work/, which for one genome is a few GB and grows with the sample sheet.
      
+
      Disk goes first. The container is 12ย GB unpacked. The eggNOG database is built on first use and cached, and it is the largest thing a run keeps: eggnog.db alone unpacks to 41.4ย GB from a 6.8ย GB download. For its bacteria-only Diamond index AMAP also downloads the 9.5ย GB eggNOG proteome, and deletes it once the index is built. Nextflow keeps every intermediate under work/, which came to 97ย MB for the test in stepย 2 and 248ย MB for all the runs of this page, none with eggNOG, and grows with the sample sheet.
      
 
      Memory decides whether the run finishes at all. Two steps are heavy:
      
-
      StepCPUsMemory requested
      eggNOG annotation2450 GB
      eggNOG database build1645 GB
      Pangenome1616 GB
      Duplication analysis6 to 1310 GB
      Report18 GB
      Bakta annotation126 GB
      Everything else1 to 86 GB or less
      
-
      Nextflow will not start a task that asks for more memory than the machine has, so on a laptop these requests are the thing that fails, not your data. Lower the ceiling and every request is clamped to it:
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --max_cpus 4 --max_memory 12.GB
      
-
      eggNOG will be slower and may still run out on a genuinely small machine. The alternative is to skip functional annotation altogether:
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --skip_functional_annotation
      
-
      You still get the corrected genes, which is what AMAP exists to produce, as a table with their proteins and sequences, and amap_report.html. What you lose is eggNOG categories, the GFF3 and GenBank files written after it, the pangenome, the duplication analysis and final_report.html. The heaviest step left asks for 6 GB.
      
-
      Docker Desktop has its own memory limit, and it is the one that counts
      On macOS and Windows, Docker runs your containers inside a virtual machine with a memory ceiling of its own, often 8 GB by default no matter how much the machine has. Nextflow sizes its requests against the host, so it will happily start a 12 GB task that the VM then kills. You get exit code 137 and nothing that mentions Docker.
      
-
      Raising --max_memory makes this worse, not better. Raise the VM instead, in Docker Desktop under Settings, Resources, Memory, and keep --max_memory at or below what you set there.
      
+
      StepCPUsMemory requested
      eggNOG annotation2450ย GB
      eggNOG database build1645ย GB
      Pangenome1616ย GB
      Duplication analysis6ย toย 1310ย GB
      Report18ย GB
      Bakta annotation126ย GB
      Everything else1ย toย 86ย GB or less
      
+
      Nextflow will not start a task that asks for more memory than the machine has, so on a laptop these requests are the thing that fails, not your data. Lower the ceiling to what docker info reports, and every request is clamped to it. Leave eggNOG out as well: it loads its whole database into memory, about 45ย GB, whatever the ceiling, so no cap makes it fit.
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --max_cpus 4 --max_memory 8.GB \
+    --skip_functional_annotation
      
+
      You still get the corrected genes, which is what AMAP exists to produce, as a table with their proteins and sequences, and amap_report.html. What you lose is eggNOG categories, the GFF3 and GenBank files written after it, the pangenome, the duplication analysis and final_report.html. The heaviest step left asks for 6ย GB.
      
+
      CautionDocker Desktop has its own memory limit, and it is the one that counts
      On macOS and Windows, Docker runs your containers inside a virtual machine with a memory ceiling of its own, often 8ย GB by default no matter how much the machine has. Nextflow sizes its requests against the host, so it will happily start a 12ย GB task that the VM then kills. You get exit code 137 and nothing that mentions Docker.
      
+
      Raising --max_memory makes this worse, not better. Raise the VM instead, in Docker Desktop under Settings, Resources, Memory, and keep --max_memory at or below what you set there.
      
 
      Linux has no such layer. Containers use host memory directly and this does not apply.
      
-
      2. Check the plumbing before touching real data#
      
-
      The test profile annotates the reference genome that ships inside the container, so it needs no data of your own:
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker,test
      
-
      The first run pulls the AMAP image. It is around 3.4 GB compressed and 12 GB on disk, because it carries the Bakta database and the H37Rv reference files. That is a one-off; later runs reuse it.
      
-
      If this finishes without an error, the whole chain works.
      
-
      3. Get a genome#
      
+
      2. Check the plumbing before touching real data
      #
      
+
      The test profile annotates the H37Rv genome that ships inside the container, so it needs no data of your own, and --skip_functional_annotation keeps it to the correction, with no eggNOG database to build:
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker,test \
+    --skip_functional_annotation --outdir amap-test
      
+
      The first run pulls the AMAP image. It is around 3.4ย GB compressed and 12ย GB on disk, because it carries the Bakta database and the H37Rv reference files. That is a one-off; later runs reuse it. With the image already pulled, the test took 2ย min 47ย s on the machine above, and wrote amap-test/ and work/ in the folder it ran from.
      
+
      If the whole chain works, it ends like this, and then names the work directory and the command:
      
+
      Output
      =======================================
+โœ“ COMPLETED
+=======================================
+Pipeline    : PathoGenOmics-Lab/AMAP v1.0.1
+Nextflow    : 26.10.2
+Profile     : docker,test
+Duration    : 2m 47s
+Succeeded   : 10
+Failed      : 0
+Ignored     : 0
+Output      : amap-test
      
+
      3. Get aย genome
      #
      
 
      Make a working directory and fetch an assembly:
      
-
      mkdir amap-tutorial && cd amap-tutorial
-
-NCBI=https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF
-ASM=000/008/585/GCF_000008585.1_ASM858v1/GCF_000008585.1_ASM858v1_genomic
-
-curl -sL -o cdc1551.fna.gz "$NCBI/$ASM.fna.gz"
-gunzip cdc1551.fna.gz
      
+
      Terminal
      $ mkdir amap-tutorial && cd amap-tutorial
+
+$ NCBI=https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF
+$ ASM=000/008/585/GCF_000008585.1_ASM858v1/GCF_000008585.1_ASM858v1_genomic
+
+$ curl -sL -o cdc1551.fna.gz "$NCBI/$ASM.fna.gz"
+$ gunzip cdc1551.fna.gz
      
 
      Have a look at what you got:
      
-
      grep '^>' cdc1551.fna
      
-
      >NC_002755.2 Mycobacterium tuberculosis CDC1551, complete sequence
      
-
      One closed contig, 4,403,837 bp. AMAP takes assembled genomes, so reads are not an input here.
      
-
      4. Write the sample sheet#
      
+
      Terminal
      $ grep '^>' cdc1551.fna
      
+
      Output
      >NC_002755.2 Mycobacterium tuberculosis CDC1551, complete sequence
      
+
      One closed contig, 4,403,837ย bp. AMAP takes assembled genomes, so reads are not an input here.
      
+
      4. Write the sample sheet
      #
      
 
      A tab-separated file with exactly two columns:
      
-
      printf 'sampleId\tfasta\nCDC1551\t%s/cdc1551.fna\n' "$PWD" > samples.tsv
-cat samples.tsv
      
-
      sampleId	fasta
+
      Terminal
      $ printf 'sampleId\tfasta\nCDC1551\t%s/cdc1551.fna\n' "$PWD" > samples.tsv
+$ cat samples.tsv
      
+
      Output
      sampleId	fasta
 CDC1551	/path/to/amap-tutorial/cdc1551.fna
      
-
      It has to be a real tab
      Spaces will not do, and this is the single most common reason a sample sheet is rejected. Check with cat -t samples.tsv: real tabs show up as ^I. (cat -A does the same on Linux but is not accepted by the cat that ships with macOS.)
      
-
      Keep sample IDs to letters, digits and underscores. Several scripts cut the name at the first dot, so G26689.1 becomes G26689.
      
-
      5. Validate before committing to a long run#
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --validate_only
      
-
      The checks write their verdict to results/pipeline_info/validation.log, which ends:
      
-
      tail -n 8 results/pipeline_info/validation.log
      
-
      --- FASTA Validation (1 samples) ---
-
-============================================================
-Results: 1 OK, 0 warnings, 0 failed
-Total: 0 error(s), 0 warning(s)
-============================================================
-
-  โœ“ PASSED โ€” all 1 samples look good
      
+
      CautionIt has to be a real tab
      Spaces will not do, and this is the single most common reason a sample sheet is rejected. Check with cat -t samples.tsv: real tabs show up as ^I. (cat -A does the same on Linux but is not accepted by the cat that ships with macOS.)
      
+
      Keep sample IDs to letters, digits and underscores. The classification cuts names at the first dot, so a sample G26689.1 would be set aside and missing from the results.
      
+
      5. Validate before committing to aย long run
      #
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --validate_only
      
+
      Checks that pass write their verdict to results/pipeline_info/validation.log; one that fails stops the run and prints the whole report instead. The verdict is two lines:
      
+
      Terminal
      $ grep -E '^(Results|Total)' results/pipeline_info/validation.log
      
+
      Output
      Results: 1 OK, 0 warnings, 0 failed
+Total: 0 error(s), 0 warning(s)
      
+
      The log's last line then starts โœ“ PASSED.
      
 
      This costs seconds and catches the mistakes that would otherwise surface minutes in. Preparing inputs lists every check and its threshold.
      
-
      6. Run it#
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
      
-
      What happens, in order: lineage typing, Bakta annotation, alignment of the H37Rv proteome onto your genome, case classification, correction, eggNOG functional annotation, duplication analysis, pangenome, report.
      
-
      The first run also downloads and builds the eggNOG database, which is several gigabytes and dominates the wall clock. Point --eggnog_db at somewhere persistent so the next run reuses it:
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --eggnog_db ~/databases/eggnog
      
-
      If the run dies partway, -resume picks up from the last completed step.
      
-
      7. Look at what came out#
      
-
      ls -F results/
      
-
      AMAP/  BAKTA/  CDS_CASES/  DUPLICATIONS/  PANGENE/  PATHOTYPR/
-amap_report.html  final_report.html  pipeline_info/
      
-
      A run without eggNOG, like the ones behind this page, has no DUPLICATIONS/, PANGENE/ or final_report.html.
      
-
      Open results/final_report.html first. It is self-contained, so it travels by email. results/amap_report.html goes through the same run question by question, and a run without eggNOG still gets it.
      
-
      Did the correction do anything?#
      
-
      column -t -s$'\t' results/CDS_CASES/summary/*-summary.tsv
      
-
      Type           Number
-case2          2813     coordinates matched the reference exactly
-case3            24     coordinates matched, alignment length differed
-case4          1100     overlapped the reference but disagreed, resolved
-case1_initial   141     Bakta called it, the reference has nothing there
-rv_missing       97     the reference has a gene there, Bakta missed it
-total_cds      4077     what Bakta called before correction
-case1_removed   130     "novel" genes that were not really novel
-case1_kept       11     kept as genuinely absent from H37Rv
      
-
      CDC1551 against H37Rv4,077 Bakta calls
      case 2 2,813case 3 24case 4 1,100case 1, dropped 130case 1, kept 11missing, put back 97
      The same numbers, drawn: most of Bakta's calls agree with H37Rv, about a quarter overlap it with other coordinates and are settled, and very few are new.
      
-
      Read that as: of 4077 Bakta calls, 2813 already agreed with the reference, 1100 had to be adjudicated, and 97 reference genes Bakta had missed were rescued. Only 11 of the 141 apparently novel genes survived scrutiny, which is what you would expect for a strain of the same species as the reference.
      
-
      Sanity check the sample#
      
-
      column -t -s$'\t' results/AMAP/*/*-AMAP.qc.tsv
      
+
      6. Run it
      #
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
      
+
      What happens, in order: lineage typing, Bakta annotation, alignment of the H37Rv proteome onto your genome, case classification, correction, eggNOG functional annotation, duplication analysis, pangenome, the two reports.
      
+
      The first run also downloads and builds the eggNOG database, eggnog.db alone 41.4ย GB, and that dominates the wall clock. Point --eggnog_db at somewhere persistent so the next run reuses it:
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --eggnog_db ~/databases/eggnog
      
+
      If the run dies partway, -resume picks up from the last completed step.
      
+
      How long it took
      #
      
+
      Without eggNOG, on the machine above, the CDC1551 run took 2ย min 16ย s from start to finish. Each step, from the run's execution trace in results/pipeline_info/, its realtime and peak_rss:
      
+
      StepTimePeak memory
      PATHOTYPR, lineage typing1.1ย s6.1ย MB
      BAKTA_ANNOTATOR1ย min 46ย s3.5ย GB
      MINIPROT2ย s324ย MB
      TYPE_CASES, the classification14.2ย s219.2ย MB
      FIX_CDS, the correction3.7ย s165.2ย MB
      AMAP_REPORT944ย ms35.6ย MB
      
+
      The run's other four tasks, which set up the reference and record the parameters, the versions and the validation, took under 4ย s each. Bakta is the step to plan for: the E.ย coli run of stepย 9 took 5ย min 47ย s, of which Bakta on Sakai took 2ย min 28ย s at 5.7ย GB, and PREPARE_REFERENCE, which runs Bakta on the reference itself, 2ย min 36ย s.
      
+
      7. Look at what came out
      #
      
+
      Terminal
      $ ls -1F results/
      
+
      Output
      AMAP/
+BAKTA/
+CDS_CASES/
+PATHOTYPR/
+amap_report.html
+pipeline_info/
      
+
      That is a run without eggNOG, like the ones behind this page. A full run adds DUPLICATIONS/, PANGENE/ and final_report.html.
      
+
      Open results/amap_report.html first: every run writes it, and it goes through the run question by question. After a full run, results/final_report.html has the whole analysis. Both are self-contained, so they travel by email.
      
+
      Did the correction do anything?
      #
      
+
      Every file of the sample carries its name as lineage typing leaves it, L4_CDC1551. The case summary's first two columns are the counts:
      
+
      Terminal
      $ cut -f1,2 results/CDS_CASES/summary/L4_CDC1551-summary.tsv | column -t
      
+
      Output
      Type           Number
+case2          2813
+case3          24
+case4          1100
+case1_initial  141
+rv_missing     97
+total_cds      4077
+case1_removed  130
+case1_kept     11
      
+
      The other three, Sequence, Lineage and Sample, say whose counts they are, which matters once the folder holds several samples. What each row means:
      
+
      CaseCountWhat it means
      case22,813Coordinates matched the reference exactly
      case324Coordinates matched, the alignment's length differed
      case41,100Overlapped the reference but disagreed, and was resolved
      case1_initial141Bakta called it, the reference has nothing there
      rv_missing97The reference has a gene there, Bakta missed it
      total_cds4,077What Bakta called before the correction
      case1_removed130"Novel" genes that were not really novel
      case1_kept11Kept as genuinely absent from H37Rv
      
+
      CDC1551 against H37Rv4,077 Bakta calls
      caseย 2 2,813caseย 3 24caseย 4 1,100caseย 1, dropped 130caseย 1, kept 11missing, put back 97
      The same numbers, drawn: most of Bakta's calls agree with H37Rv, about a quarter overlap it with other coordinates and are settled, and very few are new.
      
+
      Read that as: of 4,077 Bakta calls, 2,813 already agreed with the reference, 1,100 had to be adjudicated, and 97 reference genes Bakta had missed were rescued. Only 11 of the 141 apparently novel genes survived scrutiny, which is what you would expect for a strain of the same species as the reference.
      
+
      Cases 2, 3, 4 and the initial caseย 1 add up to 4,078, one more than the calls. One call, L4_CDC1551_03643, has the same ends as two reference proteins: Rv3478 at its own length, which makes it caseย 2, and Rv1361c, a PPE protein three residues longer, which makes it caseย 3 too.
      
+
      Sanity-check the sample
      #
      
+
      Terminal
      $ column -t -s$'\t' results/AMAP/*/*-AMAP.qc.tsv
      
 
      Six of its eleven columns:
      
-
      genome_size  n_contigs  gc_pct  total_cds  pseudogene_count  coding_density_pct
+
      Output
      genome_size  n_contigs  gc_pct  total_cds  pseudogene_count  coding_density_pct
 4403837      1          65.61   4014       123               91.3
      
 
      65.6% GC and 91.3% coding density are what a healthy MTBC genome looks like. A number well outside that is worth investigating before trusting the rest.
      
-
      This table is written after eggNOG, so a run without it has none. The line above was made by running that step's own script, GET_OUTPUTS, on the corrected genes of the run behind this page, with an eggNOG table that has no rows. None of the six columns depends on eggNOG; the one that does, eggnog_annotation_pct, is left out.
      
-
      The files you will actually use#
      
-
      Lineage typing puts the lineage in front of the sample ID, so CDC1551 comes out as L4_CDC1551:
      
-
      WhatWhere
      Corrected annotationresults/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gff3
      Same, GenBankresults/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gbk
      Proteins for downstream toolsresults/AMAP/L4_CDC1551/L4_CDC1551-AMAP.clean.faa
      Everything known per generesults/AMAP/L4_CDC1551/L4_CDC1551-AMAPmastertable.tsv
      Tool versions for your methodsresults/pipeline_info/software_versions.yml
      
-
      Two protein files, and they differ on purpose
      *-AMAP.faa has one entry per annotated gene, pseudogenes included, so it matches the GFF3 gene for gene. *-AMAP.clean.faa drops pseudogenes and anything with an internal stop codon, which is what orthology, phylogenomics and dN/dS tools expect. It is legitimately shorter.
      
+
      This table is written after eggNOG, so a run without it has none. The line above was made by running that step's own script, GET_OUTPUTS, on the corrected genes of the run behind this page, with an eggNOG table that has no rows. None of the six columns depends on eggNOG; the one that does, eggnog_annotation_pct, is left out.
      
+
      The files you will actually use
      #
      
+
      Lineage typing puts the lineage in front of the sample ID, so CDC1551 comes out as L4_CDC1551:
      
+
      WhatWhereWritten
      Corrected annotationresults/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gff3after eggNOG
      Same, GenBankresults/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gbkafter eggNOG
      Proteins for downstream toolsresults/AMAP/L4_CDC1551/L4_CDC1551-AMAP.clean.faaafter eggNOG
      Everything known per generesults/AMAP/L4_CDC1551/L4_CDC1551-AMAPmastertable.tsvafter eggNOG
      Tool versions for your methodsresults/pipeline_info/software_versions.ymlevery run
      
+
      The run above skipped eggNOG, so in their place it has L4_CDC1551.AMAP.tsv, L4_CDC1551.AMAP.faa and L4_CDC1551.AMAP.fna, with a dot: the corrected genes, their proteins and their sequences.
      
+
      NoteTwo protein files, and they differ on purpose
      *-AMAP.faa has one entry per annotated gene, pseudogenes included, so it matches the GFF3 gene for gene. *-AMAP.clean.faa drops pseudogenes and anything with an internal stop codon, which is what orthology, phylogenomics and dN/dS tools expect. It is legitimately shorter.
      
 
      Every directory is documented under outputs.
      
-
      8. More than one genome#
      
+
      8. More than one genome
      #
      
 
      Add rows. The pipeline parallelises across samples, and the pangenome and the cross-genome duplication analysis only become meaningful with several:
      
-
      sampleId	fasta
-CDC1551	/data/cdc1551.fna
-Erdman	/data/erdman.fna
-Beijing_1	/data/beijing_1.fna
      
-
      Past a couple of dozen, move to a cluster:
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
-    --tsv samples.tsv --eggnog_db /shared/databases/eggnog
      
-
      Deployment on HPC covers scratch, shared filesystems and where things should live.
      
-
      9. The same thing for another organism#
      
+
      Tab separated
      sampleId	fasta
+CDC1551	/data/cdc1551.fna
+Erdman	/data/erdman.fna
+Beijing_1	/data/beijing_1.fna
      
+
      Past about twenty genomes, move to a cluster:
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
+    --tsv samples.tsv --eggnog_db /shared/databases/eggnog
      
+
      HPC clusters covers scratch, shared filesystems and where things should live.
      
+
      9. The same thing for another organism
      #
      
 
      Everything above assumed MTBC. For anything else you supply the trusted reference yourself, and turn off the two MTBC-specific steps.
      
-
      Fetch a reference and a sample. Here, E. coli O157:H7 Sakai corrected against K-12 MG1655:
      
-
      NCBI=https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF
-K12=000/005/845/GCF_000005845.2_ASM584v2/GCF_000005845.2_ASM584v2_genomic
-SAKAI=000/008/865/GCF_000008865.2_ASM886v2/GCF_000008865.2_ASM886v2_genomic
-
-# the reference: annotation plus the genome it describes
-curl -sL -o ref.gbk.gz   "$NCBI/$K12.gbff.gz"
-curl -sL -o ref.fna.gz   "$NCBI/$K12.fna.gz"
-
-# the sample
-curl -sL -o sakai.fna.gz "$NCBI/$SAKAI.fna.gz"
-
-gunzip ref.gbk.gz ref.fna.gz sakai.fna.gz
-printf 'sampleId\tfasta\nSakai\t%s/sakai.fna\n' "$PWD" > ecoli.tsv
      
-
      nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv ecoli.tsv \
-    --skip_typing --skip_mycobrowser \
-    --genus Escherichia --species "Escherichia coli" \
-    --ref_annotation ref.gbk --ref_genome ref.fna
      
-
      --skip_typing because lineage typing is specific to tuberculosis, and --skip_mycobrowser because those functional categories are an MTBC resource.
      
+
      Fetch a reference and a sample. Here, E.ย coli O157:H7 Sakai corrected against K-12 MG1655:
      
+
      Terminal
      $ NCBI=https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF
+$ K12=000/005/845/GCF_000005845.2_ASM584v2/GCF_000005845.2_ASM584v2_genomic
+$ SAKAI=000/008/865/GCF_000008865.2_ASM886v2/GCF_000008865.2_ASM886v2_genomic
+
+# the reference: annotation plus the genome it describes
+$ curl -sL -o ref.gbk.gz   "$NCBI/$K12.gbff.gz"
+$ curl -sL -o ref.fna.gz   "$NCBI/$K12.fna.gz"
+
+# the sample
+$ curl -sL -o sakai.fna.gz "$NCBI/$SAKAI.fna.gz"
+
+$ gunzip ref.gbk.gz ref.fna.gz sakai.fna.gz
+$ printf 'sampleId\tfasta\nSakai\t%s/sakai.fna\n' "$PWD" > ecoli.tsv
      
+
      Terminal
      $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv ecoli.tsv \
+    --skip_typing --skip_mycobrowser \
+    --genus Escherichia --species "Escherichia coli" \
+    --ref_annotation ref.gbk --ref_genome ref.fna
      
+
      --skip_typing because lineage typing is specific to tuberculosis, and --skip_mycobrowser because those functional categories are an MTBC resource.
      
 
      Before annotating anything, AMAP extracts the proteome from your reference, annotates that same reference genome with Bakta, and takes the difference to work out which genes Bakta invents. For MTBC that list was curated by hand; here it is derived automatically. On this pair it found 142.
      
-
      The case counts come out differently from the MTBC run, and the difference is biology rather than a problem:
      
-
      case2          3324
-case3            31
-case4           507
-case1_kept     1387    genes O157:H7 has and K-12 does not
-rv_missing      133
      
-
      Those 1387 are the prophages and the LEE island that O157:H7 carries and K-12 lacks. Against a reference of your own species you should expect a large case 1 when the strains genuinely differ, and a small one when they do not.
      
-
      The annotation and the genome must be the same assembly
      Coordinates from the annotation are used directly against the FASTA. An annotation from a different assembly produces nonsense: proteins that do not translate, and a nearly empty reference proteome. AMAP writes how many proteins it extracted into the work directory, where grep -h 'Curated proteins' work/*/*/.command.out finds it (4318 on this pair), so check the number looks like a genome's worth.
      
+
      The case counts come out differently from the MTBC run, and the difference is biology rather than a problem. Without lineage typing the sample keeps its name:
      
+
      Terminal
      $ cut -f1,2 results/CDS_CASES/summary/Sakai-summary.tsv | column -t
      
+
      Output
      Type           Number
+case2          3324
+case3          31
+case4          507
+case1_initial  1506
+rv_missing     133
+total_cds      5367
+case1_removed  122
+case1_kept     1387
      
+
      case1_kept counts the genes O157:H7 has and K-12 does not. Those 1,387 are the prophages and the LEE island that O157:H7 carries and K-12 lacks. Against a reference of your own species you should expect a large caseย 1 when the strains genuinely differ, and a small one when they do not.
      
+
      WarningThe annotation and the genome must be the same assembly
      Coordinates from the annotation are used directly against the FASTA. An annotation from a different assembly produces nonsense: proteins that do not translate, and a nearly empty reference proteome. AMAP writes how many proteins it extracted into the work directory, where grep -h 'Curated proteins' work/*/*/.command.out finds it (4,318 on this pair), so check the number looks like a genome's worth.
      
 
      Custom reference goes into what makes a good reference and what changes when you use one.
      
-
      When something goes wrong#
      
+
      When something goes wrong
      #
      
 
      Nextflow prints the work directory of the failed task. Everything you need is in it:
      
-
      work/ab/cd1234.../
+
      work/ab/cd1234.../
 โ”œโ”€โ”€ .command.sh    the exact script that ran
 โ”œโ”€โ”€ .command.err   stderr on its own, almost always the answer
 โ””โ”€โ”€ .exitcode
      
 
      Troubleshooting is organised by the error you are looking at.
      
-
      Where to go next#
      
-
      • Parameters: every flag and its default
      • How it works: what the correction actually does, and the thresholds behind it
      • Outputs: every file the run produces
      
-
      Edit this page on GitHub
      
-
+
      Next steps
      #
      
+
      • Parameters: every flag and its default
      • Overview: what the correction actually does, and the thresholds behind it
      • Outputs: every file the run produces
      
+
      Back to the top
      
+
      Edit this page on GitHub
      
+
 
      
-
 
 
-
 
 
 
diff --git a/docs/getting-started/tutorial.md b/docs/getting-started/tutorial.md
index 4ba2837..f17c1e2 100644
--- a/docs/getting-started/tutorial.md
+++ b/docs/getting-started/tutorial.md
@@ -2,19 +2,30 @@
 
 Every command spelled out, nothing assumed. It uses a real genome you can
 download in one line, so you can follow along and compare your numbers against
-the ones printed here. Those numbers come from actual runs of these commands,
-not from memory, made on 18 October 2026 with AMAP at commit `947a99d`. The
-runs skipped eggNOG, with `--skip_functional_annotation`. None of the numbers
-depends on it, and the one table shown here that is written after it was made
-by hand from the same run, as [the sample check](#sanity-check-the-sample)
-explains.
+the ones printed here. They come from actual runs of these commands, made on
+18 October 2026 with AMAP at commit `947a99d`. For exactly these numbers, add
+this to each `nextflow run`:
+
+```text
+-r 947a99dee49d2a2429e014296ac35ba08e911108
+```
+
+[Pinning the pipeline](../user-guide/running.md#pinning-the-pipeline) says why
+the whole hash.
+
+The runs skipped eggNOG, with `--skip_functional_annotation`, and no number
+here depends on it. The one table eggNOG's step writes, in
+[the sample check](#sanity-check-the-sample), was made by hand from the same
+run.
 
 The genome is *Mycobacterium tuberculosis* CDC1551, corrected against the H37Rv
 reference that ships inside the AMAP container. At the end there is a second
-pass showing the same thing for a non-MTBC organism.
+pass showing the same thing for an organism outside the *Mycobacterium
+tuberculosis* complex (MTBC).
 
-The correction itself takes minutes on one genome. What dominates the first run
-is setup: pulling a 12 GB image and building the eggNOG database, both of which
+The correction itself takes minutes on one genome: 2 min 16 s here, without
+eggNOG. What dominates the first full run is setup: pulling the image, 3.4 GB
+compressed and 12 GB on disk, and building the eggNOG database, both of which
 happen once and are reused afterwards. Expect the first run to take
 considerably longer than every run after it, and do not read a long silence
 during the database build as a failure.
@@ -40,19 +51,28 @@ nextflow -version
 ```
 
 > [!WARNING]
-> **Java 8 is not enough**
+> **Nextflow needs Java 17 or newer**
 >
-> Nextflow will refuse to start on Java 8 with a message about an
-> unsupported class file version. Install a newer JDK, for example
+> On Java 8 or 11, Nextflow stops with "Cannot find Java or it's a wrong
+> version". Install a newer JDK, for example
 > `conda install -c conda-forge openjdk=17`.
 
 Then a container engine. Docker on a laptop, Singularity or Apptainer on a
-cluster:
+cluster. With Docker, ask it what your containers can have:
 
 ```bash
-docker --version        # or: singularity --version
+docker info --format 'Docker can give containers {{.NCPU}} CPUs and {{.MemTotal}} bytes'
 ```
 
+```text
+Docker can give containers 14 CPUs and 8217014272 bytes
+```
+
+That is the machine behind this page: an Apple M4 Pro, with Docker Desktop
+given 8 GB of its 24. The command fails when the Docker daemon is not
+running, which `docker --version` does not catch. Under 8 GB? Raise it in
+Docker Desktop under Settings, Resources, Memory. On a cluster, `singularity --version`.
+
 Full details, including running without a container, are on the
 [installation page](installation.md).
 
@@ -74,10 +94,12 @@ Worth checking before you start, because this is what stops a run rather than
 anything you will have got wrong.
 
 Disk goes first. The container is 12 GB unpacked. The eggNOG database is built
-on first use and cached: AMAP skips the 60 GB Diamond core that eggNOG-mapper
-would otherwise fetch and builds a bacteria-only index of about 3 GB instead,
-on top of the metadata it downloads. Nextflow keeps every intermediate under
-`work/`, which for one genome is a few GB and grows with the sample sheet.
+on first use and cached, and it is the largest thing a run keeps: `eggnog.db`
+alone unpacks to 41.4 GB from a 6.8 GB download. For its bacteria-only Diamond
+index AMAP also downloads the 9.5 GB eggNOG proteome, and deletes it once the
+index is built. Nextflow keeps every intermediate under `work/`, which came to
+97 MB for the test in step 2 and 248 MB for all the runs of this page, none
+with eggNOG, and grows with the sample sheet.
 
 Memory decides whether the run finishes at all. Two steps are heavy:
 
@@ -93,19 +115,14 @@ Memory decides whether the run finishes at all. Two steps are heavy:
 
 Nextflow will not start a task that asks for more memory than the machine has,
 so on a laptop these requests are the thing that fails, not your data. Lower
-the ceiling and every request is clamped to it:
+the ceiling to what `docker info` reports, and every request is clamped to it.
+Leave eggNOG out as well: it loads its whole database into memory, about 45 GB,
+whatever the ceiling, so no cap makes it fit.
 
 ```bash
 nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --max_cpus 4 --max_memory 12.GB
-```
-
-eggNOG will be slower and may still run out on a genuinely small machine. The
-alternative is to skip functional annotation altogether:
-
-```bash
-nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --skip_functional_annotation
+    --tsv samples.tsv --max_cpus 4 --max_memory 8.GB \
+    --skip_functional_annotation
 ```
 
 You still get the corrected genes, which is what AMAP exists to produce, as a
@@ -132,18 +149,37 @@ left asks for 6 GB.
 
 ## 2. Check the plumbing before touching real data
 
-The test profile annotates the reference genome that ships inside the
-container, so it needs no data of your own:
+The test profile annotates the H37Rv genome that ships inside the container,
+so it needs no data of your own, and `--skip_functional_annotation` keeps it to
+the correction, with no eggNOG database to build:
 
 ```bash
-nextflow run PathoGenOmics-Lab/AMAP -profile docker,test
+nextflow run PathoGenOmics-Lab/AMAP -profile docker,test \
+    --skip_functional_annotation --outdir amap-test
 ```
 
 The first run pulls the AMAP image. It is around 3.4 GB compressed and 12 GB
 on disk, because it carries the Bakta database and the H37Rv reference files.
-That is a one-off; later runs reuse it.
-
-If this finishes without an error, the whole chain works.
+That is a one-off; later runs reuse it. With the image already pulled, the test
+took 2 min 47 s on the machine above, and wrote `amap-test/` and `work/` in
+the folder it ran from.
+
+If the whole chain works, it ends like this, and then names the work
+directory and the command:
+
+```text output
+=======================================
+โœ“ COMPLETED
+=======================================
+Pipeline    : PathoGenOmics-Lab/AMAP v1.0.1
+Nextflow    : 26.10.2
+Profile     : docker,test
+Duration    : 2m 47s
+Succeeded   : 10
+Failed      : 0
+Ignored     : 0
+Output      : amap-test
+```
 
 ## 3. Get a genome
 
@@ -194,8 +230,9 @@ CDC1551	/path/to/amap-tutorial/cdc1551.fna
 > `^I`. (`cat -A` does the same on Linux but is not accepted by the `cat`
 > that ships with macOS.)
 
-Keep sample IDs to letters, digits and underscores. Several scripts cut the
-name at the first dot, so `G26689.1` becomes `G26689`.
+Keep sample IDs to letters, digits and underscores. The classification cuts
+names at the first dot, so a sample `G26689.1` would be set aside and missing
+from the results.
 
 ## 5. Validate before committing to a long run
 
@@ -204,24 +241,21 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
     --tsv samples.tsv --validate_only
 ```
 
-The checks write their verdict to `results/pipeline_info/validation.log`, which
-ends:
+Checks that pass write their verdict to `results/pipeline_info/validation.log`;
+one that fails stops the run and prints the whole report instead. The verdict
+is two lines:
 
 ```bash
-tail -n 8 results/pipeline_info/validation.log
+grep -E '^(Results|Total)' results/pipeline_info/validation.log
 ```
 
 ```text
---- FASTA Validation (1 samples) ---
-
-============================================================
 Results: 1 OK, 0 warnings, 0 failed
 Total: 0 error(s), 0 warning(s)
-============================================================
-
-  โœ“ PASSED โ€” all 1 samples look good
 ```
 
+The log's last line then starts `โœ“ PASSED`.
+
 This costs seconds and catches the mistakes that would otherwise surface
 minutes in. [Preparing inputs](inputs.md) lists every check and its threshold.
 
@@ -233,10 +267,10 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
 
 What happens, in order: lineage typing, Bakta annotation, alignment of the
 H37Rv proteome onto your genome, case classification, correction, eggNOG
-functional annotation, duplication analysis, pangenome, report.
+functional annotation, duplication analysis, pangenome, the two reports.
 
-The first run also downloads and builds the eggNOG database, which is several
-gigabytes and dominates the wall clock. Point `--eggnog_db` at somewhere
+The first run also downloads and builds the eggNOG database, `eggnog.db` alone
+41.4 GB, and that dominates the wall clock. Point `--eggnog_db` at somewhere
 persistent so the next run reuses it:
 
 ```bash
@@ -246,50 +280,97 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
 
 If the run dies partway, `-resume` picks up from the last completed step.
 
+### How long it took
+
+Without eggNOG, on the machine above, the CDC1551 run took 2 min 16 s from
+start to finish. Each step, from the run's execution trace in
+`results/pipeline_info/`, its `realtime` and `peak_rss`:
+
+| Step | Time | Peak memory |
+| :--- | ---: | ---: |
+| `PATHOTYPR`, lineage typing | 1.1 s | 6.1 MB |
+| `BAKTA_ANNOTATOR` | 1 min 46 s | 3.5 GB |
+| `MINIPROT` | 2 s | 324 MB |
+| `TYPE_CASES`, the classification | 14.2 s | 219.2 MB |
+| `FIX_CDS`, the correction | 3.7 s | 165.2 MB |
+| `AMAP_REPORT` | 944 ms | 35.6 MB |
+
+The run's other four tasks, which set up the reference and record the
+parameters, the versions and the validation, took under 4 s each. Bakta is the
+step to plan for: the *E. coli* run of step 9 took 5 min 47 s, of which Bakta
+on Sakai took 2 min 28 s at 5.7 GB, and `PREPARE_REFERENCE`, which runs Bakta on
+the reference itself, 2 min 36 s.
+
 ## 7. Look at what came out
 
 ```bash
-ls -F results/
+ls -1F results/
 ```
 
 ```text
-AMAP/  BAKTA/  CDS_CASES/  DUPLICATIONS/  PANGENE/  PATHOTYPR/
-amap_report.html  final_report.html  pipeline_info/
+AMAP/
+BAKTA/
+CDS_CASES/
+PATHOTYPR/
+amap_report.html
+pipeline_info/
 ```
 
-A run without eggNOG, like the ones behind this page, has no `DUPLICATIONS/`,
-`PANGENE/` or `final_report.html`.
+That is a run without eggNOG, like the ones behind this page. A full run adds
+`DUPLICATIONS/`, `PANGENE/` and `final_report.html`.
 
-Open `results/final_report.html` first. It is self-contained, so it travels by
-email. `results/amap_report.html` goes through the same run question by
-question, and a run without eggNOG still gets it.
+Open `results/amap_report.html` first: every run writes it, and it goes through
+the run question by question. After a full run, `results/final_report.html`
+has the whole analysis. Both are self-contained, so they travel by email.
 
 ### Did the correction do anything?
 
+Every file of the sample carries its name as lineage typing leaves it,
+`L4_CDC1551`. The case summary's first two columns are the counts:
+
 ```bash
-column -t -s$'\t' results/CDS_CASES/summary/*-summary.tsv
+cut -f1,2 results/CDS_CASES/summary/L4_CDC1551-summary.tsv | column -t
 ```
 
 ```text
 Type           Number
-case2          2813     coordinates matched the reference exactly
-case3            24     coordinates matched, alignment length differed
-case4          1100     overlapped the reference but disagreed, resolved
-case1_initial   141     Bakta called it, the reference has nothing there
-rv_missing       97     the reference has a gene there, Bakta missed it
-total_cds      4077     what Bakta called before correction
-case1_removed   130     "novel" genes that were not really novel
-case1_kept       11     kept as genuinely absent from H37Rv
+case2          2813
+case3          24
+case4          1100
+case1_initial  141
+rv_missing     97
+total_cds      4077
+case1_removed  130
+case1_kept     11
 ```
 
+The other three, `Sequence`, `Lineage` and `Sample`, say whose counts they are,
+which matters once the folder holds several samples. What each row means:
+
+| Case | Count | What it means |
+| :--- | ---: | :--- |
+| `case2` | 2,813 | Coordinates matched the reference exactly |
+| `case3` | 24 | Coordinates matched, the alignment's length differed |
+| `case4` | 1,100 | Overlapped the reference but disagreed, and was resolved |
+| `case1_initial` | 141 | Bakta called it, the reference has nothing there |
+| `rv_missing` | 97 | The reference has a gene there, Bakta missed it |
+| `total_cds` | 4,077 | What Bakta called before the correction |
+| `case1_removed` | 130 | "Novel" genes that were not really novel |
+| `case1_kept` | 11 | Kept as genuinely absent from H37Rv |
+
 
 
-Read that as: of 4077 Bakta calls, 2813 already agreed with the reference,
-1100 had to be adjudicated, and 97 reference genes Bakta had missed were
+Read that as: of 4,077 Bakta calls, 2,813 already agreed with the reference,
+1,100 had to be adjudicated, and 97 reference genes Bakta had missed were
 rescued. Only 11 of the 141 apparently novel genes survived scrutiny, which is
 what you would expect for a strain of the same species as the reference.
 
-### Sanity check the sample
+Cases 2, 3, 4 and the initial case 1 add up to 4,078, one more than the calls.
+One call, `L4_CDC1551_03643`, has the same ends as two reference proteins:
+Rv3478 at its own length, which makes it case 2, and Rv1361c, a PPE protein
+three residues longer, which makes it case 3 too.
+
+### Sanity-check the sample
 
 ```bash
 column -t -s$'\t' results/AMAP/*/*-AMAP.qc.tsv
@@ -297,7 +378,7 @@ column -t -s$'\t' results/AMAP/*/*-AMAP.qc.tsv
 
 Six of its eleven columns:
 
-```text
+```text output
 genome_size  n_contigs  gc_pct  total_cds  pseudogene_count  coding_density_pct
 4403837      1          65.61   4014       123               91.3
 ```
@@ -316,13 +397,17 @@ rows. None of the six columns depends on eggNOG; the one that does,
 Lineage typing puts the lineage in front of the sample ID, so CDC1551 comes
 out as `L4_CDC1551`:
 
-| What | Where |
-| :--- | :--- |
-| Corrected annotation | `results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gff3` |
-| Same, GenBank | `results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gbk` |
-| Proteins for downstream tools | `results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.clean.faa` |
-| Everything known per gene | `results/AMAP/L4_CDC1551/L4_CDC1551-AMAPmastertable.tsv` |
-| Tool versions for your methods | `results/pipeline_info/software_versions.yml` |
+| What | Where | Written |
+| :--- | :--- | :--- |
+| Corrected annotation | `results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gff3` | after eggNOG |
+| Same, GenBank | `results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.gbk` | after eggNOG |
+| Proteins for downstream tools | `results/AMAP/L4_CDC1551/L4_CDC1551-AMAP.clean.faa` | after eggNOG |
+| Everything known per gene | `results/AMAP/L4_CDC1551/L4_CDC1551-AMAPmastertable.tsv` | after eggNOG |
+| Tool versions for your methods | `results/pipeline_info/software_versions.yml` | every run |
+
+The run above skipped eggNOG, so in their place it has `L4_CDC1551.AMAP.tsv`,
+`L4_CDC1551.AMAP.faa` and `L4_CDC1551.AMAP.fna`, with a dot: the corrected
+genes, their proteins and their sequences.
 
 > [!NOTE]
 > **Two protein files, and they differ on purpose**
@@ -346,14 +431,14 @@ Erdman	/data/erdman.fna
 Beijing_1	/data/beijing_1.fna
 ```
 
-Past a couple of dozen, move to a cluster:
+Past about twenty genomes, move to a cluster:
 
 ```bash
 nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
     --tsv samples.tsv --eggnog_db /shared/databases/eggnog
 ```
 
-[Deployment on HPC](../deployment/hpc.md) covers scratch, shared filesystems
+[HPC clusters](../deployment/hpc.md) covers scratch, shared filesystems
 and where things should live.
 
 ## 9. The same thing for another organism
@@ -397,18 +482,27 @@ work out which genes Bakta invents. For MTBC that list was curated by hand;
 here it is derived automatically. On this pair it found 142.
 
 The case counts come out differently from the MTBC run, and the difference is
-biology rather than a problem:
+biology rather than a problem. Without lineage typing the sample keeps its
+name:
+
+```bash
+cut -f1,2 results/CDS_CASES/summary/Sakai-summary.tsv | column -t
+```
 
 ```text
+Type           Number
 case2          3324
-case3            31
-case4           507
-case1_kept     1387    genes O157:H7 has and K-12 does not
-rv_missing      133
+case3          31
+case4          507
+case1_initial  1506
+rv_missing     133
+total_cds      5367
+case1_removed  122
+case1_kept     1387
 ```
 
-Those 1387 are the prophages and the LEE island that O157:H7 carries and K-12
-lacks. Against a reference of your own species you should expect a large case 1
+`case1_kept` counts the genes O157:H7 has and K-12 does not. Those 1,387 are
+the prophages and the LEE island that O157:H7 carries and K-12 lacks. Against a reference of your own species you should expect a large case 1
 when the strains genuinely differ, and a small one when they do not.
 
 > [!WARNING]
@@ -418,7 +512,7 @@ when the strains genuinely differ, and a small one when they do not.
 > annotation from a different assembly produces nonsense: proteins that do
 > not translate, and a nearly empty reference proteome. AMAP writes how many
 > proteins it extracted into the work directory, where
-> `grep -h 'Curated proteins' work/*/*/.command.out` finds it (4318 on this
+> `grep -h 'Curated proteins' work/*/*/.command.out` finds it (4,318 on this
 > pair), so check the number looks like a genome's worth.
 
 [Custom reference](../user-guide/custom-reference.md) goes into what makes a
@@ -439,8 +533,8 @@ work/ab/cd1234.../
 [Troubleshooting](../deployment/troubleshooting.md) is organised by the error
 you are looking at.
 
-## Where to go next
+## Next steps
 
 - [Parameters](../user-guide/parameters.md): every flag and its default
-- [How it works](../how-it-works/index.md): what the correction actually does, and the thresholds behind it
+- [Overview](../how-it-works/index.md): what the correction actually does, and the thresholds behind it
 - [Outputs](../user-guide/outputs.md): every file the run produces
diff --git a/docs/how-it-works.html b/docs/how-it-works.html
index e048577..bd18824 100644
--- a/docs/how-it-works.html
+++ b/docs/how-it-works.html
@@ -3,23 +3,24 @@
 
 
 
-How AMAP decides, step by step
-
+How AMAP decides, step by step | AMAP
+
 
+
 
 
-
-
+
+
 
 
 
 
 
 
-
+
 
 
-
+
 
 
 
@@ -27,16 +28,16 @@
 
      
   
      
     
-      AMAP
+      AMAP
     
     
   
      
@@ -45,39 +46,72 @@
 
      
   
      
     
      
+      
      Docs / How it works
      
       
      How AMAP decides
      
-      
      One gene at a time, through the seven steps of the correction. Pick a story, then step through it or play it. Every gene here is a real one, from runs on genomes whose right answer is known; point at a gene for its coordinates.
      
+      
      One gene at a time, in seven steps from Bakta's call to the annotation. Pick a story, then step through it or play it. Every gene here is a real one, from runs on genomes whose right answer is known; point attap a gene for its coordinates.
      
     
      
   
      
 
   
      
     
      
-      
      
+      
      
+        
      
+          
      The five outcomes, in eight stories
      
+          
      
+            Agrees Case 2
+            Same ends, other length Case 3
+            Starts late Case 4
+            One gene, two calls Case 4
+            Much longer Case 4
+            Not in the reference Case 1
+            A known false call Case 1
+            Missed by Bakta Missing
+          
      
+        
      
+        
      
+          
      The harder ones
      
+          
      
+            On its paralog's locus Paralogs
+            Too small for miniprot Exact search
+            Cut by a contig end Partial
+            Runs into the next gene Pseudogene
+            Two real copies Copies
+          
      
+        
      
+      
      
 
       
      
-        
        
+        
          
+        
          2. 
+        
          3. 
+        
          4. 
+        
          5. 
+        
          6. 
+        
          7. 
+        
          8. 
+        
        
 
         
        
-          
-          
        
+          
+          
        M. tuberculosis CDC1551, against H37Rv
        
           
        
           
        
             Bakta's call
-            reference protein, aligned
+            H37Rv protein, aligned
             what AMAP keeps
-            dropped
-            pseudogene
-          
        
-          
        
-            
        
-            
        
-            
        
-            
+            same coordinates
           
        
           
        
-            
-            
-            
+            
+            
+            
+            
+          
        
+          
        
+            
        Step 1 of 7 ยท Agrees
        
+            
        Bakta calls the genes
        
+            
        Bakta calls dnaN from 2,052 to 3,260 on the plus strand.
        
+            
           
        
         
        
       
        
@@ -91,6 +125,7 @@ 
        
       
        Go deeper
        
       
        The same steps, with every threshold and every file.
        
       
        
+        OverviewWhat goes into a decision, and the five outcomes.
         The correction algorithmEvery rule and threshold, drawn on real genes.
         The workflowEach step of the pipeline, on the metro map.
         OutputsEvery folder and file, and what a healthy run looks like.
@@ -98,6 +133,7 @@ 
        Go deeper
        
         Custom referenceAny organism, against a trusted annotation of your own.
         Duplication metricsWhat the network numbers mean.
       
        
+      
     
        
   
        
 
        
@@ -106,7 +142,7 @@ 
        Go deeper
        
   
        
     
        
       
        Cite
        
-      
        Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software].
        
+      
        Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software].
        
       
        Software: doi:10.5281/zenodo.8024446
        
     
        
     
        
@@ -116,7 +152,7 @@ 
        Learn more
        
         
      1. Documentation
        2. 
         
      2. How it works
        3. 
         
      3. Parameters
        4. 
-        
      4. Changelog
        5. 
+        
      5. Changelog
        6. 
       
     
        
     
        
@@ -125,10 +161,10 @@ 
        Project
        
         
      6. GitHub
        7. 
         
      7. Report a bug
        8. 
         
      8. Docker image
        9. 
-        
      9. GPL-3.0 · PathoGenOmics Lab
        10. 
       
     
        
   
        
+  
        PathoGenOmics LabGPL-3.0AMAP 1.0.1
        
 
 
 
diff --git a/docs/how-it-works/correction-algorithm.html b/docs/how-it-works/correction-algorithm.html
index 9c2738c..cb32417 100644
--- a/docs/how-it-works/correction-algorithm.html
+++ b/docs/how-it-works/correction-algorithm.html
@@ -4,178 +4,217 @@
 
 
 The correction algorithm | AMAP docs
-
+
 
+
 
 
 
-
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
        
   
        
     
-      AMAP
+      AMAP
     
     
   
        
 
        
 
        
-
+  
+  
+
        
 
        
-
        Docs / How it works
        
+
        Docs / How it works
        Describes AMAP on main, ahead of 1.0.1: what changed
        
 
        The correction algorithm
        
-
        The core of AMAP. It referees between the initial Bakta annotation and the Miniprot alignments of a trusted reference proteome, classifying every candidate gene and then applying a set of rules to resolve the disagreements.
        
-
        Coordinates throughout are 1-based inclusive, so a gene's length is end - start + 1. Every comparison is strand-aware and contig-aware: genes on opposite strands at the same position are different genes and never conflict, and nor are genes on different contigs of a draft assembly, where every contig starts at 1. Bakta runs with --keep-contig-headers so that it names the contigs as the FASTA and Miniprot do; by default it renames them contig_N.
        
-
        0. Placements#
        
-
        Small proteins#
        
-
        Miniprot needs a seed of several residues and enough score to report an alignment, which a protein of a few dozen residues rarely gives it: annotating E. coli K-12 with its own proteome, it placed none of 33 proteins of 8 to 29 residues, and it put the toxin IbsC (b4665), 19 residues three of which differ from its neighbour IbsD's, on that neighbour only. Every protein without an exact, complete placement is looked for in the six-frame translation of the genome, preceded by a start codon and followed by a stop, and each exact copy found where Miniprot did not already place the protein becomes a complete placement. Proteins shorter than 8 residues are not searched.
        
-
        E. coli K-12 against its own annotation. miniprot placed IbsC (b4665), 19 residues, only on its neighbour IbsD's locus; the exact search finds it here, between fau and serA, where Bakta called nothing.
        
-
        A protein of up to 50 residues with no exact copy is looked for again, allowing substitutions in at most a fifth of its residues after the first, which varies with the start codon; a stop codon in place of a residue counts as one, and the gene is then flagged as a pseudogene in step 7. E. coli O157:H7 Sakai carries ten of K-12's micro-proteins with one to four residues changed, and none was annotated. A copy with an insertion or a deletion is not found this way.
        
-
        Pieces of cut genes#
        
-
        A draft assembly cuts the genes its contig breaks run through. Miniprot reports only the placements within 5% of a protein's best, so it places the larger piece of such a gene and not the smaller one at the end of another contig: in CDC1551 cut into 40 contigs, none of the 25 smaller pieces was annotated. A placement was cut by its contig when it lacks residues at one end of its protein, ends within 30 bp of the contig's edge on that side, and those residues would not fit in the bases left. What it lacks, if 8 residues or more, is aligned again as a protein of its own, and its best placement is the piece at the other contig end; this pass runs with -L 10 -n 2, so that Miniprot indexes and chains the short reading frames an edge leaves. A placement of that protein which runs off a contig edge on the side of the cut, within 30 bp, is a piece of the gene and becomes a placement of the gene's protein. In the CDC1551 draft that places 22 of the 25 smaller pieces; the other three are five or six codons long.
        
-
        The other piece of Rv1239c in the CDC1551 draft: its last 144 residues, placed by the second pass at the very end of contig_9. Bakta's short call over it is case 4, and the piece wins.
        
-
        Paralog placements#
        
-
        Miniprot places a reference protein everywhere it scores within 5% of its best, so a protein with a paralog is placed at both loci. Before anything is classified, each placement is checked against where the protein belongs.
        
-
        The reference proteome carries each protein's coordinates in its header (Rv1150_1278269-1278820), but those are reference coordinates, and any indel upstream shifts every position after it. So each placement's reference position is predicted from the nearest proteins placed only once, which follow the sample's local offset from the reference, and the placement closest to where the protein belongs is its syntenic one. A locus that several proteins are placed at, exactly, predicts nothing: it is a repeat the reference has in several copies and the sample in one, like the single IS6110 of M. bovis, and the reference positions of its proteins are spread over the whole genome. A placement is then dropped when all of these hold:
        
+
        Every rule and threshold, drawn on real genes.
        
+
        14 min read
        
+
        On this page
        
+
+
        The core of AMAP. It referees between the initial Bakta annotation and the miniprot alignments of a trusted reference proteome, classifying every candidate gene and then applying a set of rules to resolve the disagreements.
        
+
        Coordinates throughout are 1-based inclusive, so a gene's length is end - start + 1. Every comparison is strand-aware and contig-aware: genes on opposite strands at the same position are different genes and never conflict, and neither do genes on different contigs of a draft assembly, where every contig starts at 1. Bakta runs with --keep-contig-headers so that it names the contigs as the FASTA and miniprot do; by default it renames them contig_N.
        
+
        The steps in order:
        
+
        1. Placements: small proteins found exactly, the pieces of cut genes placed, and a protein's placements on its paralogs' loci dropped.
        2. Classification: every Bakta call sorted into a case, and the reference genes Bakta missed.
        3. Filtering caseย 1: a call with nothing in the reference stays only if Bakta is not known to invent it.
        4. Resolving caseย 4: the reference's coordinates win, unless Bakta's call is much longer.
        5. Deduplication on insert: a gene displaces one already accepted only when it is much longer.
        6. Rescuing missing genes: a reference gene no gene accounts for goes back in.
        7. Final deduplication: one place per gene, plus real copies and the pieces of cut genes.
        8. Stop codon check: every gene translated, then extended, flagged as a pseudogene or marked partial.
        
+
        0. Placements
        #
        
+
        Small proteins
        #
        
+
        miniprot needs a seed of several residues and enough score to report an alignment, which a protein of a few dozen residues rarely gives it: annotating E.ย coli K-12 with its own proteome, it placed none of 33 proteins of 8ย toย 29ย residues, and it put the toxin IbsC (b4665), 19ย residues three of which differ from its neighbour IbsD's, on that neighbour only. Every protein without an exact, complete placement is looked for in the six-frame translation of the genome, preceded by a start codon and followed by a stop, and each exact copy found where miniprot did not already place the protein becomes a complete placement. Proteins shorter than 8ย residues are not searched.
        
+
        E. coli K-12 against its own annotation. miniprot placed IbsC (b4665), 19ย residues, only on its neighbour IbsD's locus; the exact search finds it here, between fau and serA, where Bakta called nothing.
        
+
        A protein of up to 50 residues with no exact copy is looked for again, allowing substitutions in at most a fifth of its residues after the first, which varies with the start codon; a stop codon in place of a residue counts as one, and the gene is then flagged as a pseudogene in stepย 7. E.ย coli O157:H7 Sakai carries ten of K-12's micro-proteins with one to four residues changed, and none was annotated. A copy with an insertion or a deletion is not found this way.
        
+
        Pieces of cut genes
        #
        
+
        A gene a contig end cuts is placed in pieces. A draft assembly cuts the genes its contig breaks run through, and miniprot reports only the placements within 5% of a protein's best, so it places the larger piece of such a gene and not the smaller one at the end of another contig: in CDC1551 cut into 40 contigs, none of the 25 smaller pieces was annotated.
        
+
        A placement was cut by its contig when all of these hold:
        
+
        • it lacks residues at one end of its protein
        • it ends within 30ย bp of the contig's edge on that side
        • those residues would not fit in the bases left
        
+
        What it lacks, if 8 residues or more, is aligned again as a protein of its own, and its best placement is the piece at the other contig end. This pass runs with -L 10 -n 2, so that miniprot indexes and chains the short reading frames an edge leaves. A placement of that protein which runs off a contig edge on the side of the cut, within 30ย bp, is a piece of the gene and becomes a placement of the gene's protein. In the CDC1551 draft that places 22 of the 25 smaller pieces; the other three are five or six codons long.
        
+
        The other piece of Rv1239c in the CDC1551 draft: its last 144ย residues, placed by the second pass at the very end of contig_9. Bakta's short call over it is caseย 4, and the piece wins.
        
+
        Paralog placements
        #
        
+
        miniprot places a reference protein everywhere it scores within 5% of its best, so a protein with a paralog is placed at both loci. Before anything is classified, each placement is checked against where the protein belongs.
        
+
        The reference proteome carries each protein's coordinates in its header (Rv1150_1278269-1278820), but those are reference coordinates, and any indel upstream shifts every position after it. So each placement's reference position is predicted from the nearest proteins placed only once, which follow the sample's local offset from the reference, and the placement closest to where the protein belongs is its syntenic one. A locus that several proteins are placed at, exactly, predicts nothing: it is a repeat the reference has in several copies and the sample in one, like the single IS6110 of M.ย bovis, and the reference positions of its proteins are spread over the whole genome. A placement is then dropped when all of these hold:
        
 
        • another protein's syntenic placement covers at least 50% of it, on the same strand and in the same reading frame
        • that protein matches there at least as well
        • the two proteins live in different places in the reference, so they are not simply neighbours that overlap
        
-
        What this removes is a protein sitting on someone else's locus: Rv1945 on Rv1148c, the mce3 genes M. bovis has lost (RD7) on their mce1 paralogs, or the sixteen other IS6110 names on each IS6110 copy. What it leaves alone is a copy at a locus no protein claims, such as an IS6110 insertion at a site the reference lacks or a duplication in the sample, and genes that genuinely overlap, which sit in different frames (Rv1041c and Rv1042c).
        
-
        CDC1551 near 400 kb: Rv0336's own locus. Its paralog Rv0515 is placed here too, and that placement is dropped.
        
-
        Near 608 kb: Rv0515's own locus, where Rv0336's placement is dropped.
        
+
        What this removes is a protein sitting on someone else's locus: Rv1945 on Rv1148c, the mce3 genes M.ย bovis has lost (RD7) on their mce1 paralogs, or the sixteen other IS6110 names on each IS6110 copy. What it leaves alone is a copy at a locus no protein claims, such as an IS6110 insertion at a site the reference lacks or a duplication in the sample, and genes that genuinely overlap, which sit in different frames (Rv1041c and Rv1042c).
        
+
        CDC1551 near 400ย kb: Rv0336's own locus. Its paralog Rv0515 is placed here too, and that placement is dropped.
        
+
        Near 608ย kb: Rv0515's own locus, where Rv0336's placement is dropped.
        
 
        When two proteins both look syntenic at one locus, which happens when a deletion removes a gene and leaves its paralog next door as the nearest place left, the better match keeps the locus. CDC1551 lacks Rv3427c and Rv3428c, and without this Rv3425 and Rv3426 would sit on Rv3429.
        
-
        1. Classification#
        
-
        Every Bakta CDS is compared against the Miniprot alignments.
        
-
        Case 2, perfect match. Same start, same end, same strand as an alignment, and the same length. Correct, kept as is.
        
-
        Case 2 in CDC1551: dnaN, where Bakta and the aligned protein agree base for base.
        
-
        Case 3, imperfect match. Same coordinates, but the aligned length differs, typically from indels. Reliable enough, kept.
        
-
        Case 3: fhaA, with the same ends as Rv0020c but six residues fewer.
        
-
        Case 4, conflicting overlap. A Bakta CDS and an alignment overlap on the same strand, but their coordinates do not match. This is the conflict that has to be adjudicated.
        
-
        Case 1, potentially novel. A Bakta CDS with no overlap to any alignment. Either a genuine gene absent from the reference, or something Bakta invented.
        
+
        1. Classification
        #
        
+
        Every Bakta CDS is compared against the miniprot alignments.
        
+
        Caseย 2, perfect match. Same start, same end, same strand as an alignment, and the same length. Correct, kept as is.
        
+
        Caseย 2 in CDC1551: dnaN, where Bakta and the aligned protein agree base for base.
        
+
        Caseย 3, imperfect match. Same coordinates, but the aligned length differs, typically from indels. Reliable enough, kept.
        
+
        Caseย 3: fhaA, with the same ends as Rv0020c but six residues fewer.
        
+
        Caseย 4, conflicting overlap. A Bakta CDS and an alignment overlap on the same strand, but their coordinates do not match. This is the conflict that has to be adjudicated.
        
+
        Caseย 1, potentially novel. A Bakta CDS with no overlap to any alignment. Either a genuine gene absent from the reference, or something Bakta invented.
        
 
        Missing reference genes. An alignment with no overlapping Bakta CDS. The reference has a gene there and the annotator missed it. Alignments with no aligned bases at all are discarded first.
        
-
        Each set is written to its own table under CDS_CASES/, so the whole decision is auditable.
        
-
        2. Filtering case 1#
        
-
        Novel-gene candidates are scrutinised before being believed. A candidate is removed if either holds:
        
+
        Each set is written to its own table under CDS_CASES/, so the whole decision is auditable.
        
+
        2. Filtering caseย 1
        #
        
+
        Novel-gene candidates are scrutinised before being believed. A candidate is dropped if either holds:
        
 
        • it overlaps the false-positive set by 80% or more of its own length, meaning it is a call the annotator is known to make wrongly on the reference genome itself
        • it aligns to the reference at 80% or more match identity, meaning it is not novel at all but a shifted version of a gene that is already there
        
-
        Candidates that pass both, and candidates with no reference alignment to judge by, are kept as case1_kept.
        
-
        Case 1 removed: a call that Bakta's known false positives cover end to end.
        
-
        Case 1 kept: four calls in a stretch CDC1551 has and H37Rv lacks.
        
-
        3. Resolving case 4#
        
-
        Single overlaps#
        
+
        Candidates that pass both, and candidates with no reference alignment to judge by, are kept as case1_kept.
        
+
        Caseย 1 dropped: a call that Bakta's known false positives cover end to end.
        
+
        Caseย 1 kept: four calls in a stretch CDC1551 has and H37Rv lacks.
        
+
        3. Resolving caseย 4
        #
        
+
        Single overlaps
        #
        
 
        One Bakta CDS against one reference alignment, on the same strand of the same contig. Lengths are compared, and:
        
-
        • the reference wins by default
        • the Bakta version wins only if it is longer by more than 550 bp, and the alignment is not complete in the call's reading frame
        
-
        A single overlap: Bakta's dnaA starts 33 bp late, and the reference's start is kept.
        
-
        The asymmetry is deliberate. A reference alignment is usually the better boundary, but a much longer Bakta call normally means the alignment fragmented, and discarding a full-length gene in favour of a partial alignment would be the worse error. A complete alignment, one that covers the whole reference protein down to its stop codon, has not fragmented: a longer call in its frame ends at the same stop and only starts further upstream (plcD, Rv1755c, 843 bp in H37Rv and 1545 bp in Bakta's call on CDC1551). A call in another frame is another gene, and keeps its chance to win on length; the alignment then comes back in step 5. A call that wins in the alignment's frame is the protein's gene, longer than in the reference, and takes its name in step 5. If the Bakta CDS overlaps more than one alignment, the reference wins regardless.
        
-
        A much longer call: 3,303 bp past an alignment that stops short of its protein's end, so Bakta's call is kept, and it takes the protein's name, Rv2024c.
        
-
        Multiple overlaps#
        
+
        • the reference wins by default
        • the Bakta version wins only if it is longer by more than 550ย bp, and the alignment is not complete in the call's reading frame
        
+
        A single overlap: Bakta's dnaA starts 33ย bp late, and the reference's start is kept.
        
+
        The asymmetry is deliberate. A reference alignment is usually the better boundary, but a much longer Bakta call normally means the alignment fragmented, and discarding a full-length gene in favour of a partial alignment would be the worse error. Beyond length:
        
+
        • A complete alignment has not fragmented. It covers the whole reference protein down to its stop codon, so a longer call in its frame ends at the same stop and only starts further upstream (plcD, Rv1755c, 843ย bp in H37Rv and 1,545ย bp in Bakta's call on CDC1551).
        • A call in another frame is another gene, and keeps its chance to win on length; the alignment then comes back in stepย 5.
        • A call that wins in the alignment's frame is the protein's gene, longer than in the reference, and takes its name in stepย 5.
        • A Bakta CDS that overlaps more than one alignment loses: the reference wins regardless.
        
+
        A much longer call: 3,303ย bp past an alignment that stops short of its protein's end, so Bakta's call is kept, and it takes the protein's name, Rv2024c.
        
+
        Multiple overlaps
        #
        
 
        Several Bakta CDS against one alignment, the signature of a gene that has been split. The best-overlapping Bakta CDS is picked first, then compared to the alignment on length:
        
-
        • the reference wins if it is at least as long, if the Bakta CDS beats it by less than 500 bp, if the alignment is complete in the call's frame, or if the call overlaps other alignments as well, which makes it several reference genes read as one (Rv3022c and Rv3021c are one reading frame in CDC1551)
        • otherwise the Bakta CDS wins
        
-
        Multiple overlaps: two Bakta calls on Rv0007, which comes back as one gene.
        
-
        4. Deduplication on insert#
        
+
        • the reference wins if it is at least as long, if the Bakta CDS beats it by less than 500ย bp, if the alignment is complete in the call's frame, or if the call overlaps other alignments as well, which makes it several reference genes read as one (Rv3022c and Rv3021c are one reading frame in CDC1551)
        • otherwise the Bakta CDS wins
        
+
        Multiple overlaps: two Bakta calls on Rv0007, which comes back as one gene.
        
+
        4. Deduplication on insert
        #
        
 
        Each winner is checked against the genes already accepted. Where a new coordinate overlaps an existing one by 70% or more on the same strand:
        
-
        • the new gene displaces the existing one if it is longer by more than 550 bp
        • it also displaces it if the existing gene is shorter by less than that margin and is not a reference gene
        • otherwise the existing gene stands and the new coordinate is dropped
        
+
        • the new gene displaces the existing one if it is longer by more than 550ย bp
        • it also displaces it if the existing gene is shorter by less than that margin and is not a reference gene
        • otherwise the existing gene stands and the new coordinate is dropped
        
 
        A reference gene is one whose identifier is a reference protein's. The test used to look for coordinates embedded in the identifier, which are stripped as the alignments are read, so no gene qualified and a call of similar length could displace any of them: Bakta's version of Rv1614, one codon longer, replaced it.
        
-
        5. Rescuing missing genes#
        
-
        A reference protein is missing when no gene accounts for any of its placements, and then it goes back in at its best one. A gene accounts for a placement when it sits on it: same contig and strand, the same reading frame, covering at least 50% of it, and not known to be another reference gene. The frame matters because genes that really overlap sit in different frames (Rv0724A and Rv0725c); another reference gene in the same frame does not count either, since some are nested (E. coli hokC inside mokC). Placements on other genes' loci have already been dropped in step 0, so what is left to rescue sits where the gene belongs or at a locus nobody claims.
        
-
        A missing reference gene: Rv0031, put back between two of Bakta's calls.
        
-
        A gene that stands for no reference protein accounts for any placement in its frame: the Bakta call that won its locus in step 3 over a shorter alignment. When no gene carries the protein's name, that call takes it, as long as the alignment spans at least half of the protein. Otherwise the gene came out with no name and the protein was missing from the table. In M. bovis, H37Rv's mmpL6 (Rv1557) is the 3' end of a 2904 bp gene that the TbD1 deletion cut short in H37Rv; CDC1551 and M. bovis carry a 4821 bp DNA methyltransferase gene whose first 506 residues are H37Rv's Rv2024c. A remnant names nothing: the RD12 deletion of M. bovis leaves the last 67 of cyp141's (Rv3121) 400 residues at the end of a 321-codon gene that is not cyp141.
        
-
        The pieces of a cut gene are parts of its protein, not copies of it, so each is missing on its own, and one in place does not keep another out. In the CDC1551 draft a 7.5 kb Bakta call displaced the larger piece of Rv0100, which comes back here though the smaller piece is in.
        
-
        6. Final deduplication#
        
+
        5. Rescuing missing genes
        #
        
+
        A reference protein that no gene accounts for goes back in, at its best placement. A gene accounts for a placement when it sits on it:
        
+
        • on the same contig and strand
        • in the same reading frame, since genes that really overlap sit in different frames (Rv0724A and Rv0725c)
        • covering at least 50% of it
        • and not known to be another reference gene, since some are nested in the same frame (E.ย coli hokC inside mokC)
        
+
        Placements on other genes' loci have already been dropped in stepย 0, so what is left to rescue sits where the gene belongs or at a locus nobody claims.
        
+
        A missing reference gene: Rv0031, put back between two of Bakta's calls.
        
+
        A longer call takes the name of the protein it beat. A gene that stands for no reference protein accounts for any placement in its frame: the Bakta call that won its locus in stepย 3 over a shorter alignment. When no gene carries the protein's name, that call takes it, as long as the alignment spans at least half of the protein; without this the gene came out with no name and the protein was missing from the table. Two calls named this way:
        
+
        • in M.ย bovis, H37Rv's mmpL6 (Rv1557) is the 3' end of a 2,904ย bp gene that the TbD1 deletion cut short in H37Rv
        • CDC1551 and M.ย bovis carry a 4,821ย bp DNA methyltransferase gene whose first 506ย residues are H37Rv's Rv2024c
        
+
        A remnant names nothing: the RD12 deletion of M.ย bovis leaves the last 67 of cyp141's (Rv3121) 400ย residues at the end of a 321-codon gene that is not cyp141.
        
+
        The pieces of a cut gene are parts of its protein, not copies of it, so each is missing on its own, and one in place does not keep another out. In the CDC1551 draft a 7.5ย kb Bakta call displaced the larger piece of Rv0100, which comes back here though the smaller piece is in.
        
+
        6. Final deduplication
        #
        
 
        Where one gene ends up at several coordinates, entries fully contained inside a different gene are dropped, always keeping at least one entry per gene so nothing disappears entirely.
        
-
        After the stop codon check, deduplicate_rvs.py ranks the genes that share a reference protein: a non-pseudogene over a pseudogene, then the longer one, then one Bakta also called at exactly those coordinates, then the one nearer where the protein belongs by synteny. The first keeps its place. The others stay as further copies of the gene when they are at least 90% as long as the first, clear of the copies kept, and not covered by another reference gene; a pseudogene must also be 90% as long as the reference protein, which a piece of a split gene is not, and anything else is a remnant of the gene and goes. A genome can carry a gene more than once, and these copies are what the duplication analysis counts: E. coli O157:H7 Sakai keeps 34 that used to be deleted. The pieces of a cut gene stay as well, when the first gene is one of them: each is marked partial at its cut end, no two are open at the same end alone (a gene has one first piece and one last, and any number between that a contig holds whole), they are clear of each other, and together they are at most 110% of the reference gene's length. Copies and pieces only Miniprot placed are all named after their protein, so later ones get _2, _3 to keep identifiers unique.
        
-
        E. coli O157:H7 Sakai against K-12: the first of two whole copies of RhsD (b0497).
        
-
        The second copy, 1.4 Mb away, is a further copy and stays, as b0497_2.
        
-
        7. Stop codon check#
        
-
        check_cds.py translates every surviving CDS.
        
+
        After the stop codon check, one gene per reference protein comes first, and the rest stay only as real copies or pieces. deduplicate_rvs.py ranks the genes that share a reference protein:
        
+
        1. a non-pseudogene over a pseudogene
        2. then the longer one
        3. then one Bakta also called at exactly those coordinates
        4. then the one nearer where the protein belongs by synteny
        
+
        The first keeps its place. The others stay as further copies of the gene when all of these hold, and anything else is a remnant of the gene and goes:
        
+
        • at least 90% as long as the first
        • clear of the copies kept
        • not covered by another reference gene
        • for a pseudogene, also 90% as long as the reference protein, which a piece of a split gene is not
        
+
        A genome can carry a gene more than once, and these copies are what the duplication analysis counts: E.ย coli O157:H7 Sakai keeps 34 that used to be deleted.
        
+
        The pieces of a cut gene stay as well, when the first gene is one of them and all of these hold:
        
+
        • each is marked partial at its cut end
        • no two are open at the same end alone: a gene has one first piece and one last, and any number between that a contig holds whole
        • they are clear of each other
        • together they are at most 110% of the reference gene's length
        
+
        Copies and pieces only miniprot placed are all named after their protein, so later ones get _2, _3 to keep identifiers unique.
        
+
        E. coli O157:H7 Sakai against K-12: the first of two whole copies of RhsD (b0497).
        
+
        The second copy, 1.4ย Mb away, is a further copy and stays, as b0497_2.
        
+
        7. Stop codon check
        #
        
+
        check_cds.py translates every surviving CDS.
        
 
        It is given the reference proteome, which settles what a translation alone cannot.
        
-
        No terminal stop codon. Coordinates are extended to the next in-frame stop, searching 300 bp in the right direction for the strand, or as far as the reference protein says is missing when that is further. If the extension introduces internal stops or finds nothing, the original coordinates are kept and the gene is flagged as a pseudogene.
        
-
        An extension goes no further than the reference protein. A gene may be extended to a stop at most 10 residues past the end of its reference protein: that completes an alignment that stopped short. A frame that runs on further has lost its stop or shifted, and the C-terminus it would add is not the gene's; the gene keeps its extent and is flagged as a pseudogene, with the reason, as PGAP and Bakta treat a frameshift. Often that frame runs on into the next gene, whose stop is then the next one (in M. bovis Rv0590 into Rv0590A and Rv3373 into Rv3374), and extending would annotate that gene twice.
        
-
        M. bovis: Rv0590's frame runs on 67 residues past its protein, through Rv0590A, to the end of Bakta's one long call. It keeps its extent as a pseudogene, and Rv0590A is a gene of its own.
        
-
        Genes a contig cut short are partial. In a draft assembly, a gene whose frame reaches the end of its contig with no stop, from within 30 bp of it, was cut by the assembly: it runs to the last codon there and is marked 3' partial. One that opens within a codon of the contig's edge without a start codon is 5' partial, and so is one that opens there with a codon that can start a gene but whose protein begins inside its reference protein: a cut leaves whatever codon it falls before at the edge, and one in six codons inside H37Rv genes is one a gene can start with. Neither is a pseudogene. Each piece of a cut gene is annotated this way, under the gene's name.
        
-
        CDC1551 in 40 contigs: Rv1239c's frame reaches the end of contig_10 with no stop codon, and this piece of the gene, Rv1239c_2, is marked partial at its 3' end. The other piece, at the end of contig_9, is marked partial at its 5' end.
        
+
        No terminal stop codon. Coordinates are extended to the next in-frame stop, searching 300ย bp in the right direction for the strand, or as far as the reference protein says is missing when that is further. If the extension introduces internal stops or finds nothing, the original coordinates are kept and the gene is flagged as a pseudogene.
        
+
        An extension goes no further than the reference protein. A gene may be extended to a stop at most 10 residues past the end of its reference protein: that completes an alignment that stopped short. A frame that runs on further has lost its stop or shifted, and the C-terminus it would add is not the gene's; the gene keeps its extent and is flagged as a pseudogene, with the reason, as PGAP and Bakta treat a frameshift. Often that frame runs on into the next gene, whose stop is then the next one (in M.ย bovis Rv0590 into Rv0590A and Rv3373 into Rv3374), and extending would annotate that gene twice.
        
+
        M. bovis: Rv0590's frame runs on 67ย residues past its protein, through Rv0590A, to the end of Bakta's one long call. It keeps its extent as a pseudogene, and Rv0590A is a gene of its own.
        
+
        Genes a contig cut short are partial, not pseudogenes, and each piece of a cut gene is marked this way, under the gene's name:
        
+
        • 3' partial: in a draft assembly, a gene whose frame reaches the end of its contig with no stop, from within 30ย bp of it, was cut by the assembly, and runs to the last codon there.
        • 5' partial: a gene that opens within a codon of the contig's edge without a start codon, or with a codon that can start a gene but whose protein begins inside its reference protein. A cut leaves whatever codon it falls before at the edge, and one in six codons inside H37Rv genes is one a gene can start with.
        
+
        CDC1551 in 40 contigs: Rv1239c's frame reaches the end of contig_10 with no stop codon, and this piece of the gene, Rv1239c_2, is marked partial at its 3' end. The other piece, at the end of contig_9, is marked partial at its 5' end.
        
 
        Reference pseudogenes keep their extent. When the reference protein spans its annotated stretch codon for codon with no stop (PE21, Rv2099c; PPE48, Rv3022c) and the gene ends where it does, the reference annotates it without a stop, and the gene is left as it is and flagged. A reference pseudogene annotated with a partial final codon (oxyR', Rv2427A) gets that base back, since an alignment only covers whole codons.
        
 
        Clipped N-termini are restored. When a gene ends in a stop but is short of its reference protein at the start, and the codons upstream open with a start codon and translate to the missing residues, the gene is extended to cover them. A gene that already starts where its reference does is shorter somewhere inside, and is left alone.
        
-
        Internal stop codons. Flagged as a pseudogene and kept, with the flag propagated to the master table and the GFF3. *-AMAP.clean.faa excludes them.
        
+
        Internal stop codons. Flagged as a pseudogene and kept, with the flag propagated to the master table and the GFF3. *-AMAP.clean.faa excludes them.
        
 
        Untranslatable genes are dropped from the annotation table as well as the FASTA files, so the two never disagree.
        
-
        Threshold summary#
        
-
        ThresholdWhereValue
        False-positive overlapcase 1 filtering80%
        Reference match identitycase 1 filtering80%
        Bakta length advantagesingle overlaps550 bp
        Bakta length advantagemultiple overlaps500 bp
        Overlap for deduplicationon insert70%
        Length margin for displacementon insert550 bp
        Distance from the predicted positionparalog placements10 kb
        Proteins placed once, each side, for the local offsetparalog placements5
        Cover to claim another protein's placementparalog placements50%
        Shortest protein looked for exactlysmall proteins8 residues
        Longest protein looked for with substitutionssmall proteins50 residues
        Residues after the first that may differsmall proteins20%
        Distance from a contig's edge for a placement, or a piece, to be cut therepieces of cut genes30 bp
        Shortest part of a protein looked for at the other contig endspieces of cut genes8 residues
        Cover for a gene to account for a placementrescuing missing genes50%
        Share of its protein an alignment spans to name a longer callrescuing missing genes50%
        Length of a further copy, next to the firstfinal deduplication90%
        Cover by another reference gene that rules a copy outfinal deduplication50%
        Length of a gene's pieces together, next to the reference genefinal deduplication110%
        Stop codon searchquality control300 bp, or what the reference protein is missing
        Residues an extension may run past the end of the reference proteinquality control10
        Distance from a contig end within which a gene counts as cutquality control30 bp at the 3' end, a codon at the 5' end
        Identity to restore a clipped N-terminusquality control90%
        
-
        These are constants in bin/classify/, tuned on MTBC. Another organism may want different values, particularly the 550 bp length margins, which assume genes of roughly bacterial size.
        
-
        Next steps#
        
+
        Threshold summary
        #
        
+
        ThresholdWhereValue
        False-positive overlapcaseย 1 filtering80%
        Reference match identitycaseย 1 filtering80%
        Bakta length advantagesingle overlaps550ย bp
        Bakta length advantagemultiple overlaps500ย bp
        Overlap for deduplicationon insert70%
        Length margin for displacementon insert550ย bp
        Distance from the predicted positionparalog placements10ย kb
        Proteins placed once, each side, for the local offsetparalog placements5
        Cover to claim another protein's placementparalog placements50%
        Shortest protein looked for exactlysmall proteins8ย residues
        Longest protein looked for with substitutionssmall proteins50ย residues
        Residues after the first that may differsmall proteins20%
        Distance from a contig's edge for a placement, or a piece, to be cut therepieces of cut genes30ย bp
        Shortest part of a protein looked for at the other contig endspieces of cut genes8ย residues
        Cover for a gene to account for a placementrescuing missing genes50%
        Share of its protein an alignment spans to name a longer callrescuing missing genes50%
        Length of a further copy, next to the firstfinal deduplication90%
        Cover by another reference gene that rules a copy outfinal deduplication50%
        Length of a gene's pieces together, next to the reference genefinal deduplication110%
        Stop codon searchstop codon check300ย bp, or what the reference protein is missing
        Residues an extension may run past the end of the reference proteinstop codon check10
        Distance from a contig end within which a gene counts as cutstop codon check30ย bp at the 3' end, a codon at the 5' end
        Identity to restore a clipped N-terminusstop codon check90%
        
+
        These are constants in bin/classify/, tuned on the Mycobacterium tuberculosis complex (MTBC). Another organism may want different values, particularly the 550ย bp length margins, which assume genes of roughly bacterial size.
        
+
        Next steps
        #
        
 
        • The workflow: where this sits in the pipeline
        • Outputs: the case tables that record every decision
        
-
        Edit this page on GitHub
        
-
+
        Back to the top
        
+
        Edit this page on GitHub
        
+
 
        
-
 
 
-
 
 
 
diff --git a/docs/how-it-works/correction-algorithm.md b/docs/how-it-works/correction-algorithm.md
index e9e63c8..39484f0 100644
--- a/docs/how-it-works/correction-algorithm.md
+++ b/docs/how-it-works/correction-algorithm.md
@@ -1,28 +1,47 @@
 # The correction algorithm
 
 The core of AMAP. It referees between the initial Bakta annotation and the
-Miniprot alignments of a trusted reference proteome, classifying every candidate
+miniprot alignments of a trusted reference proteome, classifying every candidate
 gene and then applying a set of rules to resolve the disagreements.
 
 Coordinates throughout are 1-based inclusive, so a gene's length is
 `end - start + 1`. Every comparison is strand-aware and contig-aware: genes on
 opposite strands at the same position are different genes and never conflict,
-and nor are genes on different contigs of a draft assembly, where every contig
+and neither do genes on different contigs of a draft assembly, where every contig
 starts at 1. Bakta runs with `--keep-contig-headers` so that it names the
-contigs as the FASTA and Miniprot do; by default it renames them `contig_N`.
+contigs as the FASTA and miniprot do; by default it renames them `contig_N`.
+
+The steps in order:
+
+0. [Placements](#0-placements): small proteins found exactly, the pieces of cut
+   genes placed, and a protein's placements on its paralogs' loci dropped.
+1. [Classification](#1-classification): every Bakta call sorted into a case,
+   and the reference genes Bakta missed.
+2. [Filtering case 1](#2-filtering-case-1): a call with nothing in the
+   reference stays only if Bakta is not known to invent it.
+3. [Resolving case 4](#3-resolving-case-4): the reference's coordinates win,
+   unless Bakta's call is much longer.
+4. [Deduplication on insert](#4-deduplication-on-insert): a gene displaces one
+   already accepted only when it is much longer.
+5. [Rescuing missing genes](#5-rescuing-missing-genes): a reference gene no
+   gene accounts for goes back in.
+6. [Final deduplication](#6-final-deduplication): one place per gene, plus real
+   copies and the pieces of cut genes.
+7. [Stop codon check](#7-stop-codon-check): every gene translated, then
+   extended, flagged as a pseudogene or marked partial.
 
 ## 0. Placements
 
 ### Small proteins
 
-Miniprot needs a seed of several residues and enough score to report an
+miniprot needs a seed of several residues and enough score to report an
 alignment, which a protein of a few dozen residues rarely gives it: annotating
 *E. coli* K-12 with its own proteome, it placed none of 33 proteins of 8 to 29
 residues, and it put the toxin IbsC (b4665), 19 residues three of which differ
 from its neighbour IbsD's, on that neighbour only. Every protein without an exact,
 complete placement is looked for in the six-frame translation of the genome,
 preceded by a start codon and followed by a stop, and each exact copy found
-where Miniprot did not already place the protein becomes a complete placement.
+where miniprot did not already place the protein becomes a complete placement.
 Proteins shorter than 8 residues are not searched.
 
 
@@ -37,26 +56,31 @@ this way.
 
 ### Pieces of cut genes
 
-A draft assembly cuts the genes its contig breaks run through. Miniprot reports
-only the placements within 5% of a protein's best, so it places the larger
-piece of such a gene and not the smaller one at the end of another contig: in
-CDC1551 cut into 40 contigs, none of the 25 smaller pieces was annotated. A
-placement was cut by its contig when it lacks residues at one end of its
-protein, ends within **30 bp** of the contig's edge on that side, and those
-residues would not fit in the bases left. What it lacks, if **8** residues or
-more, is aligned again as a protein of its own, and its best placement is the
-piece at the other contig end; this pass runs with `-L 10 -n 2`, so that
-Miniprot indexes and chains the short reading frames an edge leaves. A
-placement of that protein which runs off a contig edge on the side of the cut,
-within 30 bp, is a piece of the gene and becomes a placement of the gene's
-protein. In the CDC1551 draft that places 22 of the 25 smaller pieces; the
-other three are five or six codons long.
+**A gene a contig end cuts is placed in pieces.** A draft assembly cuts the
+genes its contig breaks run through, and miniprot reports only the placements
+within 5% of a protein's best, so it places the larger piece of such a gene and
+not the smaller one at the end of another contig: in CDC1551 cut into 40
+contigs, none of the 25 smaller pieces was annotated.
+
+A placement was cut by its contig when all of these hold:
+
+- it lacks residues at one end of its protein
+- it ends within **30 bp** of the contig's edge on that side
+- those residues would not fit in the bases left
+
+What it lacks, if **8** residues or more, is aligned again as a protein of its
+own, and its best placement is the piece at the other contig end. This pass
+runs with `-L 10 -n 2`, so that miniprot indexes and chains the short reading
+frames an edge leaves. A placement of that protein which runs off a contig edge
+on the side of the cut, within 30 bp, is a piece of the gene and becomes a
+placement of the gene's protein. In the CDC1551 draft that places 22 of the 25
+smaller pieces; the other three are five or six codons long.
 
 
 
 ### Paralog placements
 
-Miniprot places a reference protein everywhere it scores within 5% of its best,
+miniprot places a reference protein everywhere it scores within 5% of its best,
 so a protein with a paralog is placed at both loci. Before anything is
 classified, each placement is checked against where the protein belongs.
 
@@ -78,7 +102,7 @@ placement is then dropped when all of these hold:
   simply neighbours that overlap
 
 What this removes is a protein sitting on someone else's locus: Rv1945 on
-Rv1148c, the mce3 genes M. bovis has lost (RD7) on their mce1 paralogs, or the
+Rv1148c, the mce3 genes *M. bovis* has lost (RD7) on their mce1 paralogs, or the
 sixteen other IS6110 names on each IS6110 copy. What it leaves alone is a copy
 at a locus no protein claims, such as an IS6110 insertion at a site the
 reference lacks or a duplication in the sample, and genes that genuinely
@@ -95,7 +119,7 @@ this Rv3425 and Rv3426 would sit on Rv3429.
 
 ## 1. Classification
 
-Every Bakta CDS is compared against the Miniprot alignments.
+Every Bakta CDS is compared against the miniprot alignments.
 
 **Case 2, perfect match.** Same start, same end, same strand as an alignment,
 and the same length. Correct, kept as is.
@@ -124,7 +148,7 @@ is auditable.
 ## 2. Filtering case 1
 
 Novel-gene candidates are scrutinised before being believed. A candidate is
-**removed** if either holds:
+**dropped** if either holds:
 
 - it overlaps the [false-positive set](../user-guide/custom-reference.md) by
   **80% or more** of its own length, meaning it is a call the annotator is known
@@ -155,14 +179,18 @@ contig. Lengths are compared, and:
 The asymmetry is deliberate. A reference alignment is usually the better
 boundary, but a much longer Bakta call normally means the alignment fragmented,
 and discarding a full-length gene in favour of a partial alignment would be the
-worse error. A complete alignment, one that covers the whole reference protein
-down to its stop codon, has not fragmented: a longer call in its frame ends at
-the same stop and only starts further upstream (plcD, Rv1755c, 843 bp in H37Rv
-and 1545 bp in Bakta's call on CDC1551). A call in another frame is another
-gene, and keeps its chance to win on length; the alignment then comes back in
-step 5. A call that wins in the alignment's frame is the protein's gene, longer
-than in the reference, and takes its name in step 5. If the Bakta CDS overlaps
-more than one alignment, the reference wins regardless.
+worse error. Beyond length:
+
+- **A complete alignment has not fragmented.** It covers the whole reference
+  protein down to its stop codon, so a longer call in its frame ends at the same
+  stop and only starts further upstream (plcD, Rv1755c, 843 bp in H37Rv and
+  1,545 bp in Bakta's call on CDC1551).
+- **A call in another frame is another gene**, and keeps its chance to win on
+  length; the alignment then comes back in step 5.
+- **A call that wins in the alignment's frame is the protein's gene**, longer
+  than in the reference, and takes its name in step 5.
+- **A Bakta CDS that overlaps more than one alignment loses**: the reference
+  wins regardless.
 
 
 
@@ -197,28 +225,36 @@ displace any of them: Bakta's version of Rv1614, one codon longer, replaced it.
 
 ## 5. Rescuing missing genes
 
-A reference protein is missing when no gene accounts for any of its
-placements, and then it goes back in at its best one. A gene accounts for a
-placement when it sits on it: same contig and strand, the **same reading
-frame**, covering at least **50%** of it, and not known to be another reference
-gene. The frame matters because genes that really overlap sit in different
-frames (Rv0724A and Rv0725c); another reference gene in the same frame does not
-count either, since some are nested (*E. coli* hokC inside mokC). Placements on
-other genes' loci have already been dropped in step 0, so what is left to rescue
-sits where the gene belongs or at a locus nobody claims.
+**A reference protein that no gene accounts for goes back in**, at its best
+placement. A gene accounts for a placement when it sits on it:
+
+- on the same contig and strand
+- in the **same reading frame**, since genes that really overlap sit in
+  different frames (Rv0724A and Rv0725c)
+- covering at least **50%** of it
+- and not known to be another reference gene, since some are nested in the same
+  frame (*E. coli* hokC inside mokC)
+
+Placements on other genes' loci have already been dropped in step 0, so what is
+left to rescue sits where the gene belongs or at a locus nobody claims.
 
 
 
-A gene that stands for no reference protein accounts for any placement in its
-frame: the Bakta call that won its locus in step 3 over a shorter alignment.
-When no gene carries the protein's name, that call takes it, as long as the
-alignment spans at least **half** of the protein. Otherwise the gene came out
-with no name and the protein was missing from the table. In *M. bovis*, H37Rv's
-mmpL6 (Rv1557) is the 3' end of a 2904 bp gene that the TbD1 deletion cut short
-in H37Rv; CDC1551 and *M. bovis* carry a 4821 bp DNA methyltransferase gene
-whose first 506 residues are H37Rv's Rv2024c. A remnant names nothing: the
-RD12 deletion of *M. bovis* leaves the last 67 of cyp141's (Rv3121) 400
-residues at the end of a 321-codon gene that is not cyp141.
+**A longer call takes the name of the protein it beat.** A gene that stands
+for no reference protein accounts for any placement in its frame: the Bakta
+call that won its locus in step 3 over a shorter alignment. When no gene
+carries the protein's name, that call takes it, as long as the alignment spans
+at least **half** of the protein; without this the gene came out with no name
+and the protein was missing from the table. Two calls named this way:
+
+- in *M. bovis*, H37Rv's mmpL6 (Rv1557) is the 3' end of a 2,904 bp gene that
+  the TbD1 deletion cut short in H37Rv
+- CDC1551 and *M. bovis* carry a 4,821 bp DNA methyltransferase gene whose first
+  506 residues are H37Rv's Rv2024c
+
+A remnant names nothing: the RD12 deletion of *M. bovis* leaves the last 67 of
+cyp141's (Rv3121) 400 residues at the end of a 321-codon gene that is not
+cyp141.
 
 The pieces of a cut gene are parts of its protein, not copies of it, so each is
 missing on its own, and one in place does not keep another out. In the CDC1551
@@ -231,21 +267,38 @@ Where one gene ends up at several coordinates, entries fully contained inside a
 different gene are dropped, always keeping at least one entry per gene so
 nothing disappears entirely.
 
-After the stop codon check, `deduplicate_rvs.py` ranks the genes that share a
-reference protein: a non-pseudogene over a pseudogene, then the longer one, then
-one Bakta also called at exactly those coordinates, then the one nearer where
-the protein belongs by synteny. The first keeps its place. The others stay as
-**further copies** of the gene when they are at least **90%** as long as the
-first, clear of the copies kept, and not covered by another reference gene; a
-pseudogene must also be 90% as long as the reference protein, which a piece of
-a split gene is not, and anything else is a remnant of the gene and goes. A genome can carry a gene more
-than once, and these copies are what the duplication analysis counts: *E. coli*
-O157:H7 Sakai keeps 34 that used to be deleted. The pieces of a cut gene stay
-as well, when the first gene is one of them: each is marked partial at its cut
-end, no two are open at the same end alone (a gene has one first piece and one
-last, and any number between that a contig holds whole), they are clear of each
-other, and together they are at most **110%** of the reference gene's length.
-Copies and pieces only Miniprot placed are all named after their protein, so
+**After the stop codon check, one gene per reference protein comes first, and
+the rest stay only as real copies or pieces.** `deduplicate_rvs.py` ranks the
+genes that share a reference protein:
+
+1. a non-pseudogene over a pseudogene
+2. then the longer one
+3. then one Bakta also called at exactly those coordinates
+4. then the one nearer where the protein belongs by synteny
+
+The first keeps its place. The others stay as **further copies** of the gene
+when all of these hold, and anything else is a remnant of the gene and goes:
+
+- at least **90%** as long as the first
+- clear of the copies kept
+- not covered by another reference gene
+- for a pseudogene, also 90% as long as the reference protein, which a piece of
+  a split gene is not
+
+A genome can carry a gene more than once, and these copies are what the
+duplication analysis counts: *E. coli* O157:H7 Sakai keeps 34 that used to be
+deleted.
+
+The pieces of a cut gene stay as well, when the first gene is one of them and
+all of these hold:
+
+- each is marked partial at its cut end
+- no two are open at the same end alone: a gene has one first piece and one
+  last, and any number between that a contig holds whole
+- they are clear of each other
+- together they are at most **110%** of the reference gene's length
+
+Copies and pieces only miniprot placed are all named after their protein, so
 later ones get `_2`, `_3` to keep identifiers unique.
 
 
@@ -271,20 +324,22 @@ protein: that completes an alignment that stopped short. A frame that runs on
 further has lost its stop or shifted, and the C-terminus it would add is not
 the gene's; the gene keeps its extent and is flagged as a pseudogene, with the
 reason, as PGAP and Bakta treat a frameshift. Often that frame runs on into the
-next gene, whose stop is then the next one (in M. bovis Rv0590 into Rv0590A and
+next gene, whose stop is then the next one (in *M. bovis* Rv0590 into Rv0590A and
 Rv3373 into Rv3374), and extending would annotate that gene twice.
 
 
 
-**Genes a contig cut short are partial.** In a draft assembly, a gene whose
-frame reaches the end of its contig with no stop, from within 30 bp of it, was
-cut by the assembly: it runs to the last codon there and is marked `3'`
-partial. One that opens within a codon of the contig's edge without a start
-codon is `5'` partial, and so is one that opens there with a codon that can
-start a gene but whose protein begins inside its reference protein: a cut
-leaves whatever codon it falls before at the edge, and one in six codons inside
-H37Rv genes is one a gene can start with. Neither is a pseudogene. Each piece
-of a cut gene is annotated this way, under the gene's name.
+**Genes a contig cut short are partial**, not pseudogenes, and each piece of a
+cut gene is marked this way, under the gene's name:
+
+- **`3'` partial**: in a draft assembly, a gene whose frame reaches the end of
+  its contig with no stop, from within 30 bp of it, was cut by the assembly,
+  and runs to the last codon there.
+- **`5'` partial**: a gene that opens within a codon of the contig's edge
+  without a start codon, or with a codon that can start a gene but whose
+  protein begins inside its reference protein. A cut leaves whatever codon it
+  falls before at the edge, and one in six codons inside H37Rv genes is one a
+  gene can start with.
 
 
 
@@ -330,12 +385,12 @@ FASTA files, so the two never disagree.
 | Length of a further copy, next to the first | final deduplication | 90% |
 | Cover by another reference gene that rules a copy out | final deduplication | 50% |
 | Length of a gene's pieces together, next to the reference gene | final deduplication | 110% |
-| Stop codon search | quality control | 300 bp, or what the reference protein is missing |
-| Residues an extension may run past the end of the reference protein | quality control | 10 |
-| Distance from a contig end within which a gene counts as cut | quality control | 30 bp at the 3' end, a codon at the 5' end |
-| Identity to restore a clipped N-terminus | quality control | 90% |
+| Stop codon search | stop codon check | 300 bp, or what the reference protein is missing |
+| Residues an extension may run past the end of the reference protein | stop codon check | 10 |
+| Distance from a contig end within which a gene counts as cut | stop codon check | 30 bp at the 3' end, a codon at the 5' end |
+| Identity to restore a clipped N-terminus | stop codon check | 90% |
 
-These are constants in `bin/classify/`, tuned on MTBC. Another organism may want
+These are constants in `bin/classify/`, tuned on the *Mycobacterium tuberculosis* complex (MTBC). Another organism may want
 different values, particularly the 550 bp length margins, which assume genes of
 roughly bacterial size.
 
diff --git a/docs/how-it-works/duplications.html b/docs/how-it-works/duplications.html
index a0abd86..f093360 100644
--- a/docs/how-it-works/duplications.html
+++ b/docs/how-it-works/duplications.html
@@ -4,119 +4,134 @@
 
 
 Duplication metrics | AMAP docs
-
+
 
+
 
 
 
-
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
        
   
        
     
-      AMAP
+      AMAP
     
     
   
        
 
        
 
        
-
+  
+  
+
        
 
        
-
        Docs / How it works
        
+
        Docs / How it works
        Describes AMAP on main, ahead of 1.0.1: what changed
        
 
        Duplication metrics
        
+
        What the network numbers mean.
        
+
        3 min read
        
+
        On this page
        
+
 
        AMAP looks at gene duplication twice: within each genome, and across the whole set.
        
-
        How the families are found#
        
-
        Each corrected proteome is searched against itself with Diamond, excluding self-hits. A pair is kept when the identical residues of the alignment cover 80% or more of either the query or the subject, so a short domain match between two otherwise unrelated proteins is not counted as duplication. Kept pairs become links in a graph, and its connected pieces become the duplication families, listed in DUPLICATIONS/{sample}/duplications_{sample}_groups.tsv.
        
-
        in the familya link: identical residues cover 80% or more of one of the twoa hit under 80%: no linka protein outside the family
        CDC1551's corrected proteome against itself, run as GET_DUPLICATIONS runs it: six transposases make one family. Each line is a pair Diamond matched, with the larger of the two shares its identical residues cover; the size of a protein is its length. Point at a protein or a line for its numbers.
        
+
        How the families are found
        #
        
+
        Each corrected proteome is searched against itself with Diamond, excluding self-hits. A pair is kept when the identical residues of the alignment cover 80% or more of either the query or the subject, so a short domain match between two otherwise unrelated proteins is not counted as duplication. Kept pairs become links in a graph, and its connected pieces become the duplication families, listed in DUPLICATIONS/{lineage}_{sampleId}/duplications_{lineage}_{sampleId}_groups.tsv.
        
+
        in the familya link: identical residues cover 80% or more of one of the twoa hit under 80%: no linka protein outside the family
        CDC1551's corrected proteome against itself, run as GET_DUPLICATIONS runs it: six transposases make one family. Each line is a pair Diamond matched, with the larger of the two shares its identical residues cover; the size of a protein is its length. Point at a protein or a line for its numbers.
        
 
        Because the rule takes the larger of the two shares, a fragment joins the family of the gene it came from: a match that covers almost all of a short protein counts, however little of the long one it covers.
        
-
        How the communities are built#
        
-
        The proteins of every sample's families are then pooled and searched all against all. Within each genome, the pairs that pass the same rule make a network, which the Louvain method splits into communities, with a fixed seed so that a rerun splits it the same way. Two communities are homologous when at least one hit joins them, and each homologous pair is one row of DUPLICATIONS/homologous_communities.tsv, with an interactive network in the matching HTML file.
        
-
        a link in this genomea link only the other genome has
        The same family in GET_ALLDUP, pooled with M. bovis AF2122/97's families: one community in each genome, joined by their hits, and the row of homologous_communities.tsv they make. Each protein of M. bovis sits where its closest match in CDC1551 does.
        
-
        The columns#
        
-
        A row describes two communities: the columns ending in 1 are the first, those ending in 2 the second. The first is the one whose sample comes first by name, or the one with the lower number when both are in the same sample, and the rows come in that order too, so a rerun writes the same file. For each:
        
-
        • Genome, Community: the sample, and the community's number in it.
        • Size: how many proteins it holds.
        • Elements: the eggNOG COG categories of its proteins, with how many have each, the largest first. A protein eggNOG gave no category counts as -, eggNOG's mark for none.
        • AvgDegree: links per protein, on average. In a community of n proteins it reaches n - 1 when every protein is linked to every other.
        • Density: the links it has as a share of the pairs it could have. 1 means every protein matches every other.
        • AvgClustering: for each protein, how often two of its partners are linked to each other as well, averaged over the community. 1 when every protein is linked to every other, 0 for a chain or a star.
        • AvgPathLength: how many links separate two of its proteins, on average, over its largest connected piece.
        • Nodes: the proteins themselves.
        
-
        Reading them together#
        
+
        How the communities are built
        #
        
+
        The proteins of every sample's families are then pooled and searched all against all. Within each genome, the pairs that pass the same rule make a network, which the Louvain method splits into communities, with a fixed seed so that a rerun splits it the same way. Two communities are homologous when at least one hit joins them, and each homologous pair is one row of DUPLICATIONS/homologous_communities.tsv, with an interactive network in the matching HTML file.
        
+
        a link in this genomea link only the other genome has
        The same family in GET_ALLDUP, pooled with M. bovis AF2122/97's families: one community in each genome, joined by their hits, and the row of homologous_communities.tsv they make. Each protein of M. bovis sits where its closest match in CDC1551 does.
        
+
        The columns
        #
        
+
        A row describes two communities: the columns ending in 1 are the first, those ending in 2 the second. The first is the one whose sample comes first by name, or the one with the lower number when both are in the same sample, and the rows come in that order too, so a rerun writes the same file. For each:
        
+
        • Genome, Community: the sample, and the community's number in it.
        • Size: how many proteins it holds.
        • Elements: the eggNOG COG categories of its proteins, with how many have each, the largest first. A protein eggNOG gave no category counts as -, eggNOG's mark for none.
        • AvgDegree: links per protein, on average. In a community of n proteins it reaches n - 1 when every protein is linked to every other.
        • Density: the links it has as a share of the pairs it could have. 1 means every protein matches every other.
        • AvgClustering: for each protein, how often two of its partners are linked to each other as well, averaged over the community. 1 when every protein is linked to every other, 0 for a chain or a star.
        • AvgPathLength: how many links separate two of its proteins, on average, over its largest connected piece.
        • Nodes: the proteins themselves.
        
+
        Reading them together
        #
        
 
        Density and clustering at 1 mean copies that all match each other: a recent expansion of near-identical copies. In CDC1551 these are the transposases of insertion sequences, the ESAT-6-like Esx proteins and PE proteins.
        
-
        Lower density and longer paths mean copies that have drifted apart, each still matching some of the others but not all, as M. bovis's copies of the transposases above have. Or, less happily, an artefact of assembly fragmentation, where one gene broken across contigs looks like several partial copies. Cross-checking against the contig count in *-AMAP.qc.tsv tells the two apart.
        
-
        Duplications and assembly quality
        Repetitive families are exactly what short-read assemblers collapse or fragment. A genome with markedly fewer duplications than its relatives may have collapsed them rather than lost them. The metrics describe the assembly you provided, not necessarily the organism.
        
-
        Next steps#
        
+
        Lower density and longer paths mean copies that have drifted apart, each still matching some of the others but not all, as M.ย bovis's copies of the transposases above have. Or, less happily, an artefact of assembly fragmentation, where one gene broken across contigs looks like several partial copies. Cross-checking against the contig count in *-AMAP.qc.tsv tells the two apart.
        
+
        NoteDuplications and assembly quality
        Repetitive families are exactly what short-read assemblers collapse or fragment. A genome with markedly fewer duplications than its relatives may have collapsed them rather than lost them. The metrics describe the assembly you provided, not necessarily the organism.
        
+
        Next steps
        #
        
 
-
        Edit this page on GitHub
        
-
+
        Edit this page on GitHub
        
+
 
        
-
 
 
-
 
 
 
diff --git a/docs/how-it-works/duplications.md b/docs/how-it-works/duplications.md
index e038c1b..d446f4d 100644
--- a/docs/how-it-works/duplications.md
+++ b/docs/how-it-works/duplications.md
@@ -11,7 +11,7 @@ self-hits. A pair is kept when the identical residues of the alignment cover
 between two otherwise unrelated proteins is not counted as duplication. Kept
 pairs become links in a graph, and its connected pieces become the
 duplication families, listed in
-`DUPLICATIONS/{sample}/duplications_{sample}_groups.tsv`.
+`DUPLICATIONS/{lineage}_{sampleId}/duplications_{lineage}_{sampleId}_groups.tsv`.
 
 
 
@@ -62,7 +62,7 @@ transposases of insertion sequences, the ESAT-6-like Esx proteins and PE
 proteins.
 
 **Lower density and longer paths** mean copies that have drifted apart, each
-still matching some of the others but not all, as M. bovis's copies of the
+still matching some of the others but not all, as *M. bovis*'s copies of the
 transposases above have. Or, less happily, an artefact of assembly
 fragmentation, where one gene broken across contigs looks like several partial
 copies. Cross-checking against the contig count in `*-AMAP.qc.tsv` tells the
diff --git a/docs/how-it-works/index.html b/docs/how-it-works/index.html
index a1657ef..64617e9 100644
--- a/docs/how-it-works/index.html
+++ b/docs/how-it-works/index.html
@@ -4,121 +4,135 @@
 
 
 Overview | AMAP docs
-
+
 
+
 
 
 
-
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
        
   
        
     
-      AMAP
+      AMAP
     
     
   
        
 
        
 
        
-
+  
+  
+
        
 
        
-
        Docs / How it works
        
-
        How it works
        
+
        Docs / How it works
        Describes AMAP on main, ahead of 1.0.1: what changed
        
+
        Overview
        
+
        What goes into a decision, and the five outcomes.
        
+
        2 min read
        
+
        On this page
        
+
 
        AMAP does not annotate genomes from scratch. It takes an annotation that already exists, compares it against one that is trusted, and adjudicates the differences.
        
 
        The premise is that de novo annotators are good at finding genes and less good at their exact boundaries. Bakta will call a start codon a few codons late, or split one gene into two, or invent a short ORF that is not really there. If a curated reference for a close relative exists, those errors are visible: align the reference proteins onto the new genome, and where the two disagree, decide which is more likely right.
        
-
        On MTBC genomes that adjudication changes roughly 30% of genes.
        
-
        The three inputs to a decision#
        
+
        On genomes of the Mycobacterium tuberculosis complex (MTBC) that adjudication changes roughly 30% of genes.
        
+
        The three inputs to aย decision
        #
        
 
        The Bakta annotation. Every CDS Bakta called on your genome, with coordinates and a locus tag.
        
-
        The Miniprot alignments. Every reference protein aligned onto your genome, with the coordinates where it landed. The reference proteins carry their own coordinates in their headers (Rv0001_1-1524), which lets a later step tell a real locus from a paralogous placement.
        
+
        The miniprot alignments. Every reference protein aligned onto your genome, with the coordinates where it landed. The reference proteins carry their own coordinates in their headers (Rv0001_1-1524), which lets a later step tell a real locus from a paralogous placement.
        
 
        The false-positive set. Proteins that a de novo annotator calls on the reference genome itself but that are not in the curated annotation. Bakta's characteristic mistakes, in other words. For MTBC this is a curated file; with a custom reference AMAP derives it by annotating the reference and taking the difference.
        
-
        The three inputs over the first 12.4 kb of CDC1551: Bakta's calls, the H37Rv proteins miniprot placed, and two of the calls Bakta makes on H37Rv itself that its curated annotation does not have, aligned here. Point at a gene for its coordinates.
        
-
        The pipeline#
        
-
        Ten steps, from genomes to report:
        
-
        1. eggNOG database setup, skipped if it is already built
        2. Lineage typing with pathotypr, MTBC only
        3. Annotation with Bakta
        4. Protein alignment with Miniprot
        5. Case classification: every CDS sorted by how it relates to the reference
        6. Correction: conflicts resolved, missing genes rescued, stop codons checked
        7. Functional annotation with eggNOG-mapper
        8. Duplication analysis: within and across genomes
        9. Pangenome with pangene
        10. Report
        
+
        The three inputs over the first 12.4ย kb of CDC1551: Bakta's calls, the H37Rv proteins miniprot placed, and two of the calls Bakta makes on H37Rv itself that its curated annotation does not have, aligned here. Point at a gene for its coordinates.
        
+
        The pipeline
        #
        
+
        Eleven steps, from genomes to reports:
        
+
        1. eggNOG database setup, skipped if it is already built
        2. Lineage typing with pathotypr, MTBC only
        3. Annotation with Bakta
        4. Protein alignment with miniprot
        5. Case classification: every CDS sorted by how it relates to the reference
        6. Correction: conflicts resolved, missing genes rescued, stop codons checked
        7. Functional annotation with eggNOG-mapper
        8. Duplication analysis: within and across genomes
        9. Pangenome with pangene
        10. The full report, final_report.html, after eggNOG
        11. The report by question, amap_report.html, in every run
        
 
        The workflow page walks through each with its inputs and outputs.
        
-
        The correction itself#
        
+
        The correction itself
        #
        
 
        Steps 5 and 6 are what makes AMAP more than a wrapper. Every CDS is sorted into one of five outcomes:
        
-
        CaseMeaning
        Case 1Bakta called it, the reference has nothing there. Potentially novel.
        Case 2Coordinates match the reference exactly. Kept.
        Case 3Coordinates match but alignment length differs. Kept.
        Case 4Overlaps a reference gene, but the coordinates conflict. Must be resolved.
        MissingThe reference has a gene there, Bakta missed it. Rescued.
        
-
        Bakta's callreference protein, alignedwhat AMAP keeps
        Case 2Perfect matchSame start, end, strand and length as an aligned protein. Kept, tied to it.Step through itCase 3Imperfect matchSame start, end and strand; the alignment's length differs. Kept, tied to the protein.The ruleCase 4Conflicting overlapOverlaps an aligned protein with other coordinates. The reference's win, unless the call is much longer.Step through itCase 1Potentially novelNothing aligns. Kept, unless Bakta is known to call it wrongly or the reference genome has its protein.Step through itMissingPut backA reference protein aligns where Bakta called nothing. It goes in at the alignment.Step through it
        And the harder ones
        The five outcomes, each on real genes of CDC1551: the same windows as the case pages, drawn small. Each card opens its story, step by step.
        
+
        CaseMeaning
        Caseย 1Bakta called it, the reference has nothing there. Potentially novel.
        Caseย 2Coordinates match the reference exactly. Kept.
        Caseย 3Coordinates match but alignment length differs. Kept.
        Caseย 4Overlaps a reference gene, but the coordinates conflict. Must be resolved.
        MissingThe reference has a gene there, Bakta missed it. Rescued.
        
+
        Bakta's callreference protein, alignedwhat AMAP keeps
        Caseย 2Perfect matchSame start, end, strand and length as an aligned protein. Kept, tied to it.Step through itCaseย 3Imperfect matchSame start, end and strand; the alignment's length differs. Kept, tied to the protein.Step through itCaseย 4Conflicting overlapOverlaps an aligned protein with other coordinates. The reference wins, unless the call is much longer.Step through itCaseย 1Potentially novelNothing aligns. Kept, unless Bakta is known to call it wrongly or the reference genome has its protein.Step through itMissingPut backA reference protein aligns where Bakta called nothing. It goes in at the alignment.Step through it
        And the harder ones
        The five outcomes, each on real genes of CDC1551: the same windows as the front page's cases and the correction algorithm's figures, drawn small. Each card opens its story, step by step.
        
 
        Cases 1 and 4 are where the work is: deciding which novel genes are real, and which set of coordinates wins a conflict. The correction algorithm gives the rules and the thresholds.
        
-
        In this section#
        
-
-
        Edit this page on GitHub
        
-
+
        Next steps
        #
        
+
+
        Edit this page on GitHub
        
+
 
        
-
 
 
-
 
 
 
diff --git a/docs/how-it-works/index.md b/docs/how-it-works/index.md
index 7a87814..d5168aa 100644
--- a/docs/how-it-works/index.md
+++ b/docs/how-it-works/index.md
@@ -1,4 +1,4 @@
-# How it works
+# Overview
 
 AMAP does not annotate genomes from scratch. It takes an annotation that already
 exists, compares it against one that is trusted, and adjudicates the
@@ -11,14 +11,14 @@ curated reference for a close relative exists, those errors are visible: align
 the reference proteins onto the new genome, and where the two disagree, decide
 which is more likely right.
 
-On MTBC genomes that adjudication changes roughly 30% of genes.
+On genomes of the *Mycobacterium tuberculosis* complex (MTBC) that adjudication changes roughly 30% of genes.
 
 ## The three inputs to a decision
 
 **The Bakta annotation.** Every CDS Bakta called on your genome, with
 coordinates and a locus tag.
 
-**The Miniprot alignments.** Every reference protein aligned onto your genome,
+**The miniprot alignments.** Every reference protein aligned onto your genome,
 with the coordinates where it landed. The reference proteins carry their own
 coordinates in their headers (`Rv0001_1-1524`), which lets a later step tell a
 real locus from a paralogous placement.
@@ -33,18 +33,19 @@ annotating the reference and taking the difference.
 
 ## The pipeline
 
-Ten steps, from genomes to report:
+Eleven steps, from genomes to reports:
 
 1. **eggNOG database setup**, skipped if it is already built
 2. **Lineage typing** with pathotypr, MTBC only
 3. **Annotation** with Bakta
-4. **Protein alignment** with Miniprot
+4. **Protein alignment** with miniprot
 5. **Case classification**: every CDS sorted by how it relates to the reference
 6. **Correction**: conflicts resolved, missing genes rescued, stop codons checked
 7. **Functional annotation** with eggNOG-mapper
 8. **Duplication analysis**: within and across genomes
 9. **Pangenome** with pangene
-10. **Report**
+10. **The full report**, `final_report.html`, after eggNOG
+11. **The report by question**, `amap_report.html`, in every run
 
 [The workflow page](workflow.md) walks through each with its inputs and outputs.
 
@@ -68,8 +69,8 @@ which set of coordinates wins a conflict.
 [The correction algorithm](correction-algorithm.md) gives the rules and the
 thresholds.
 
-## In this section
+## Next steps
 
-- [The workflow](workflow.md): the ten steps in detail
-- [The correction algorithm](correction-algorithm.md): case classification, conflict resolution, quality control
+- [The workflow](workflow.md): the eleven steps in detail
+- [The correction algorithm](correction-algorithm.md): case classification, conflict resolution, stop codon check
 - [Duplication metrics](duplications.md): what the network numbers mean
diff --git a/docs/how-it-works/workflow.html b/docs/how-it-works/workflow.html
index b2e70c3..2746891 100644
--- a/docs/how-it-works/workflow.html
+++ b/docs/how-it-works/workflow.html
@@ -4,142 +4,165 @@
 
 
 The workflow | AMAP docs
-
+
 
+
 
 
 
-
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
        
   
        
     
-      AMAP
+      AMAP
     
     
   
        
 
        
 
        
-
+  
+  
+
        
 
        
-
        Docs / How it works
        
+
        Docs / How it works
        Describes AMAP on main, ahead of 1.0.1: what changed
        
 
        The workflow
        
-
        The pipeline as a metro map: stations are the processes of main.nf, lines are what travels between them. Pick a station for what it does and where its files go, or switch an option to see what a run leaves out.
        
-
        Run with
        Scroll sideways to follow the lines.
        
-
        1. eggNOG database setup#
        
-
        Checks whether the eggNOG database exists at --eggnog_db and builds it if not. A bacteria.dmnd under 100 MB is treated as a truncated download and rebuilt.
        
-
        The database is large, so point --eggnog_db at persistent shared storage on a cluster. Skipped entirely under --skip_functional_annotation.
        
-
        2. Lineage typing#
        
-
        pathotypr assigns each genome to an MTBC lineage using phylogenetic marker SNPs, and the genomes are renamed after their sample ID with the lineage in front (L4_G26689 for sample G26689). That prefix carries through every downstream file and is what lets the report group results by lineage.
        
-
        pathotypr types each contig of a draft on its own, and the lineage of the draft is the one the markers of all its contigs add up to. A genome of one sequence has that sequence renamed after the sample and keeps the description that follows the name in its header, so Bakta still reads a closed genome as complete and circular when the description says so (NCBI's "complete genome" and "complete sequence", Unicycler's circular=true); the contigs of a draft keep their names. A genome with no lineage markers at all is named NA_G26689.
        
-
        MTBC-specific. Turn it off with --skip_typing for anything else, and sample IDs stay as you wrote them.
        
-
        3. Annotation with Bakta#
        
-
        Bakta produces the baseline annotation: CDS, tRNA, rRNA and other features. This is the annotation AMAP will correct, and it is published unmodified under BAKTA/ so you can compare.
        
-
        --genus and --species are passed through, so setting them correctly improves the starting point.
        
-
        4. Protein alignment with Miniprot#
        
-
        Miniprot aligns the reference proteome onto each genome, spliced and frameshift-aware. Two alignments are produced: the reference proteome, and the false-positive set that will filter novel-gene candidates.
        
+
        Each step of the pipeline, on the metro map.
        
+
        5 min read
        
+
        On this page
        
+
+
        The pipeline as a metro map: stations are the processes of main.nf, lines are what travels between them. Pick a station for what it does and where its files go, or switch an option to see what a run leaves out.
        
+
        Run with
        Scroll sideways to follow the lines.
        Every process of main.nf, in the order it runs them. The line under the panel lists what a run with these switches writes.
        
+
        1. eggNOG database setup
        #
        
+
        Checks whether the eggNOG database exists at --eggnog_db and builds it if not. A bacteria.dmnd under 100ย MB is treated as a truncated download and rebuilt.
        
+
        The database is large, so point --eggnog_db at persistent shared storage on a cluster. Skipped entirely under --skip_functional_annotation.
        
+
        2. Lineage typing
        #
        
+
        pathotypr assigns each genome to a lineage of the Mycobacterium tuberculosis complex (MTBC) using phylogenetic marker SNPs, and the genomes are renamed after their sample ID with the lineage in front (L4_G26689 for sample G26689). That prefix carries through every downstream file and is what lets the report group results by lineage.
        
+
        pathotypr types each contig of a draft on its own, and the lineage of the draft is the one the markers of all its contigs add up to. A genome of one sequence has that sequence renamed after the sample and keeps the description that follows the name in its header, so Bakta still reads a closed genome as complete and circular when the description says so (NCBI's "complete genome" and "complete sequence", Unicycler's circular=true); the contigs of a draft keep their names. A genome with no lineage markers at all is named NA_G26689.
        
+
        MTBC-specific. Turn it off with --skip_typing for anything else, and sample IDs stay as you wrote them.
        
+
        3. Annotation with Bakta
        #
        
+
        Bakta produces the baseline annotation: CDS, rRNA and other features (AMAP runs it with --skip-trna, so there are no tRNAs). This is the annotation AMAP will correct, and it is published unmodified under BAKTA/ so you can compare.
        
+
        --genus and --species are passed through, so setting them correctly improves the starting point.
        
+
        4. Protein alignment with miniprot
        #
        
+
        miniprot aligns the reference proteome onto each genome, spliced and frameshift-aware. Two alignments are produced: the reference proteome, and the false-positive set that will filter novel-gene candidates.
        
 
        Output is PAF. Coordinates are converted from 0-based to 1-based on read, and for a protein hitting several positions, the best hit per position is kept so multi-copy families such as PE/PPE keep all their placements.
        
-
        In a draft assembly the genes a contig end cuts get a second pass. Miniprot places the larger piece of such a gene only, so what that placement lacks is aligned again as a protein of its own, and its placement at the end of another contig is the rest of the gene (pieces of cut genes).
        
-
        5. Case classification#
        
-
        Every Bakta CDS is compared against the alignments and sorted into cases 1 to 4, plus the reference genes Bakta missed. Comparisons are strand-aware throughout: a gene at (100, 200, +) and one at (100, 200, -) are different genes and are never merged.
        
-
        Tables land in CDS_CASES/, one per case, plus interactive overlap plots. The correction algorithm covers the classification.
        
-
        6. Correction#
        
+
        In a draft assembly the genes a contig end cuts get a second pass. miniprot places the larger piece of such a gene only, so what that placement lacks is aligned again as a protein of its own, and its placement at the end of another contig is the rest of the gene (pieces of cut genes).
        
+
        5. Case classification
        #
        
+
        Every Bakta CDS is compared against the alignments and sorted into cases 1ย toย 4, plus the reference genes Bakta missed. Comparisons are strand-aware throughout: a gene at (100, 200, +) and one at (100, 200, -) are different genes and are never merged.
        
+
        Tables land in CDS_CASES/, one per case, plus interactive overlap plots. The correction algorithm covers the classification.
        
+
        6. Correction
        #
        
 
        The cases are resolved into one corrected coordinate set: conflicts adjudicated, novel genes filtered, missing reference genes rescued, duplicates removed.
        
-
        Then check_cds.py translates every surviving CDS. A gene that does not end in a stop codon has its coordinates extended to the next in-frame stop, up to 300 bp. Genes with internal stops are flagged as pseudogenes rather than discarded. Anything that cannot be translated at all is dropped, so the annotation table and the FASTA files always agree.
        
-
        Outputs are the corrected gene table and its protein and nucleotide files under AMAP/ (*.AMAP.tsv, .faa, .fna). The GFF3 and GenBank files follow once eggNOG has run, in step 7.
        
-
        7. Functional annotation#
        
+
        Then check_cds.py translates every surviving CDS. A gene that does not end in a stop codon has its coordinates extended to the next in-frame stop, up to 300ย bp. Genes with internal stops are flagged as pseudogenes rather than discarded. Anything that cannot be translated at all is dropped, so the annotation table and the FASTA files always agree.
        
+
        Outputs are the corrected gene table and its protein and nucleotide files under AMAP/ (*.AMAP.tsv, .faa, .fna). The GFF3 and GenBank files follow once eggNOG has run, in stepย 7.
        
+
        7. Functional annotation
        #
        
 
        eggNOG-mapper assigns COG categories, KEGG orthologs and pathways, GO terms, EC numbers, PFAM domains and CAZy families.
        
 
        This runs once over all samples together, not once per sample. With closely related genomes most proteins are identical across samples, so they are deduplicated first, annotated once, and the results propagated back. For a few hundred MTBC genomes that cuts the Diamond search by an order of magnitude and avoids hundreds of jobs contending for the same database on a shared filesystem.
        
-
        The results are merged into *-AMAPmastertable.tsv and into the GFF3 attributes.
        
-
        8. Duplication analysis#
        
+
        The results are merged into *-AMAPmastertable.tsv and into the GFF3 attributes.
        
+
        8. Duplication analysis
        #
        
 
        Each proteome is searched against itself with Diamond to find within-genome duplications, which are grouped into families by connected components. Then the proteins of every sample's families are pooled and searched all against all to build homologous communities across genomes, with network metrics computed per community.
        
 
        Duplication metrics explains the numbers.
        
-
        9. Pangenome#
        
+
        9. Pangenome
        #
        
 
        pangene builds one pangenome graph for the whole run and a gene presence/absence matrix, which the report turns into core, softcore, shell and cloud classifications, a frequency spectrum and presence/absence heatmaps.
        
-
        Every genome is searched with the same proteins: the reference proteome, and the genes of each sample that no reference gene matches. miniprot aligns them to every sample's genome, its own included: the ones PATHOTYPR renamed or, with --skip_typing, the genomes of the sample sheet, each under its sample ID. A row of the matrix is then a gene, named after its reference gene or, outside the reference, after one sample's copy of it, and a column a sample, so the genes the samples share are the rows found in every column.
        
-
        10. Report#
        
-
        A single self-contained HTML file built from every output above. Sample quality, missing reference genes by lineage, PE/PPE families, KEGG pathways, duplication communities and topology, the pangenome, COG and Mycobrowser distributions, case statistics with the associated tests, and a searchable table of every gene.
        
-
        Alongside it, pipeline_info/ collects the Nextflow execution report, timeline, DAG and trace, plus software_versions.yml and params.yml for your methods section.
        
-
        11. Interactive report#
        
-
        amap_report.html, one self-contained file organised by question: how each genome annotated, where reference genes are missing or broken, one gene in every genome, what the correction changed and the pseudogenes by lineage. It reads the correction's outputs directly, so it is written with or without the functional annotation, and the functional steps add their sections when they run. The interactive report goes through it.
        
-
        Next steps#
        
-
-
        Edit this page on GitHub
        
-
+
        Every genome is searched with the same proteins: the reference proteome, and the genes of each sample that no reference gene matches. miniprot aligns them to every sample's genome, its own included: the ones PATHOTYPR renamed or, with --skip_typing, the genomes of the sample sheet, each under its sample ID. A row of the matrix is then a gene, named after its reference gene or, outside the reference, after one sample's copy of it, and a column a sample, so the genes the samples share are the rows found in every column.
        
+
        10. The full report
        #
        
+
        final_report.html, a single self-contained HTML file built from every output above, so only a run with the functional annotation has it. Sample quality, missing reference genes by lineage, PE/PPE families, KEGG pathways, duplication communities and topology, the pangenome, COG and Mycobrowser distributions, case statistics with the associated tests, and a searchable table of every gene.
        
+
        Alongside it, pipeline_info/ collects the Nextflow execution report, timeline, DAG and trace, plus software_versions.yml and params.yml for your methods section.
        
+
        11. The report by question
        #
        
+
        amap_report.html, one self-contained file organised by question, written in every run: how each genome annotated, where reference genes are missing or broken, one gene in every genome, what the correction changed and the pseudogenes by lineage. It reads the correction's outputs directly, so it is written with or without the functional annotation, and the functional steps add their sections when they run. The report by question goes through it.
        
+
        Next steps
        #
        
+
+
        Back to the top
        
+
        Edit this page on GitHub
        
+
 
        
-
 
 
-
 
 
 
diff --git a/docs/how-it-works/workflow.md b/docs/how-it-works/workflow.md
index 4918013..7783d16 100644
--- a/docs/how-it-works/workflow.md
+++ b/docs/how-it-works/workflow.md
@@ -17,7 +17,7 @@ cluster. Skipped entirely under `--skip_functional_annotation`.
 ## 2. Lineage typing
 
 [pathotypr](https://github.com/PathoGenOmics-Lab/pathotypr) assigns each genome
-to an MTBC lineage using phylogenetic marker SNPs, and the genomes are renamed
+to a lineage of the *Mycobacterium tuberculosis* complex (MTBC) using phylogenetic marker SNPs, and the genomes are renamed
 after their sample ID with the lineage in front (`L4_G26689` for sample
 `G26689`). That prefix carries through every downstream file and is what lets
 the report group results by lineage.
@@ -37,15 +37,16 @@ IDs stay as you wrote them.
 ## 3. Annotation with Bakta
 
 [Bakta](https://github.com/oschwengers/bakta) produces the baseline annotation:
-CDS, tRNA, rRNA and other features. This is the annotation AMAP will correct,
+CDS, rRNA and other features (AMAP runs it with `--skip-trna`, so there are no
+tRNAs). This is the annotation AMAP will correct,
 and it is published unmodified under `BAKTA/` so you can compare.
 
 `--genus` and `--species` are passed through, so setting them correctly improves
 the starting point.
 
-## 4. Protein alignment with Miniprot
+## 4. Protein alignment with miniprot
 
-[Miniprot](https://github.com/lh3/miniprot) aligns the reference proteome onto
+[miniprot](https://github.com/lh3/miniprot) aligns the reference proteome onto
 each genome, spliced and frameshift-aware. Two alignments are produced: the
 reference proteome, and the false-positive set that will filter novel-gene
 candidates.
@@ -54,7 +55,7 @@ Output is PAF. Coordinates are converted from 0-based to 1-based on read, and
 for a protein hitting several positions, the best hit per position is kept so
 multi-copy families such as PE/PPE keep all their placements.
 
-In a draft assembly the genes a contig end cuts get a second pass. Miniprot
+In a draft assembly the genes a contig end cuts get a second pass. miniprot
 places the larger piece of such a gene only, so what that placement lacks is
 aligned again as a protein of its own, and its placement at the end of another
 contig is the rest of the gene
@@ -126,9 +127,10 @@ row of the matrix is then a gene, named after its reference gene or, outside
 the reference, after one sample's copy of it, and a column a sample, so the
 genes the samples share are the rows found in every column.
 
-## 10. Report
+## 10. The full report
 
-A single self-contained HTML file built from every output above. Sample quality,
+`final_report.html`, a single self-contained HTML file built from every output
+above, so only a run with the functional annotation has it. Sample quality,
 missing reference genes by lineage, PE/PPE families, KEGG pathways, duplication
 communities and topology, the pangenome, COG and Mycobrowser distributions, case
 statistics with the associated tests, and a searchable table of every gene.
@@ -137,14 +139,15 @@ Alongside it, `pipeline_info/` collects the Nextflow execution report, timeline,
 DAG and trace, plus `software_versions.yml` and `params.yml` for your methods
 section.
 
-## 11. Interactive report
+## 11. The report by question
 
-`amap_report.html`, one self-contained file organised by question: how each
+`amap_report.html`, one self-contained file organised by question, written in
+every run: how each
 genome annotated, where reference genes are missing or broken, one gene in
 every genome, what the correction changed and the pseudogenes by lineage. It
 reads the correction's outputs directly, so it is written with or without the
 functional annotation, and the functional steps add their sections when they
-run. [The interactive report](../user-guide/amap-report.md) goes through it.
+run. [The report by question](../user-guide/amap-report.md) goes through it.
 
 ## Next steps
 
diff --git a/docs/index.html b/docs/index.html
index 89654f4..ebb3a58 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -4,22 +4,23 @@
 
 
 AMAP: every gene Bakta calls, checked against a reference
-
+
 
+
 
 
 
-
+
 
 
 
 
 
 
-
+
 
 
-
+
 
 
 
@@ -27,42 +28,60 @@
 
        
   
        
     
-      AMAP
+      AMAP
     
     
   
        
 
        
 
+
+
 
        
-  
        
+  
        
     
        
-      
        
+      
        
         
        A Nextflow pipeline for bacterial genomes
        
         
        Bakta calls the genes. AMAP checks every one.
        
-        
        AMAP annotates assembled genomes with Bakta, aligns a trusted reference proteome onto them with miniprot, and settles every disagreement between the two: late starts, genes split in two, calls that are not genes, genes Bakta missed. Built for the Mycobacterium tuberculosis complex, where it changes about 30% of the genes, and for any species you have a reference for.
        
-        
        
-          Start here
-          Read the tutorial
-          GitHub
+        
        AMAP annotates assembled genomes with Bakta, aligns a trusted reference proteome onto them with miniprot, and settles every disagreement: late starts, split genes, calls that are not genes, genes Bakta missed. Built for the M. tuberculosis complex, and for any species with a reference.
        
+        
        
+          
        
+            Start here
+            Read the tutorial
+            GitHub
+          
        
+          
        
+            
        In
        assembled genomes (FASTA)
        
+            
        Out
        corrected GFF3, GenBank and proteome, the tables behind every decision, and a report of the whole run
        
+          
        
         
        
-        
        Reads assembled genomes. Writes, for each one, a corrected GFF3, GenBank and proteome with the tables behind every decision, and one report for the whole run.
        
       
        
 
       
        
         
        
-          M. tuberculosis CDC1551, its first 12.4 kb, against H37Rv. Point at a gene.
+          
+          
        What AMAP keeps
        
           
         
        
-        
        What AMAP keeps
        
         
        
         
        
           Bakta's call
@@ -76,82 +95,132 @@ 
        Bakta calls the genes. AMAP checks every one.
        
           
        4genes get the reference's coordinates
        
           
        3calls are dropped
        
         
        
+        
        
+          
        The first 12.4 kb of M. tuberculosis CDC1551, against H37Rv. Twelve calls by Bakta, the eight H37Rv proteins aligned there, and the eight genes AMAP keeps. Point at a gene.
        
+        
        
       
        
     
        
   
        
 
+  
        
+    
        
+      
        
+        
        Across the whole genome
        
+        
        M. tuberculosis CDC1551, against H37Rv
        
+        See each case
+      
        
+      
          
+        
        • 4,077calls by Bakta, each held up against the reference
          • 
+        
        • 1,100disagree with it, and are settled
          • 
+        
        • 130are not genes, and are dropped
          • 
+        
        • 97genes Bakta missed, put back
          • 
+      
        
+    
        
+  
        
+
   
        
     
        
       
        
         
        Start here
        
-        
        From nothing to a corrected genome
        
-        
        Every tool AMAP calls lives in one container image, so Nextflow and a container engine are all you install.
        
+        
        From nothing to a corrected genome
        
+        
        Every tool AMAP calls lives in one container image, so Nextflow and a container engine are all you install. Four steps; once the image is in, the slowest takes minutes.
        
       
        
       
          
         
        1. 
           
           
          
             
          Install Nextflow and a container engine
          
-            
          Nextflow needs Java 17 or newer, and conda brings both. Then Docker on a laptop, or Singularity or Apptainer on a cluster.
          
-            
          $ conda create -n nextflow-env -c conda-forge -c bioconda nextflow
-$ conda activate nextflow-env
-$ docker --version
          
-            
          On an Apple Silicon Mac the image runs under emulation: fine for the test below, slow for a real genome. Installation, in full.
          
+            
          Nextflow needs Java 17 or newer, and conda brings both. Then Docker on a laptop, or Singularity or Apptainer on a cluster.
          
+            
          $ conda create -n nextflow-env -c conda-forge -c bioconda nextflow
+$ conda activate nextflow-env
+$ docker info --format 'Docker can give containers {{.NCPU}} CPUs and {{.MemTotal}} bytes'
          
+            
          The last line fails if Docker is not running, and says what a container may use; Docker Desktop's 8 GB prints as 8217014272 bytes. Under 8 GB? Raise it in Docker Desktop under Settings, Resources, Memory. On Apple Silicon the image runs emulated; every timing on this site was measured that way. Installation, in full.
          
           
          
         
          2. 
         
        2. 
           
           
          
             
          Check the plumbing
          
-            
          The test profile annotates the H37Rv genome that ships inside the image, so it needs no data of your own. If it finishes without an error, the whole chain works.
          
-            
          $ nextflow run PathoGenOmics-Lab/AMAP -profile docker,test
          
-            
          The first run pulls the image: about 3.4 GB compressed, 12 GB on disk, once.
          
+            
          The test profile annotates the H37Rv genome inside the image, without eggNOG: no data, no database. Run it in an empty folder; it downloads only the image and the pipeline, and writes amap-test/ and work/. It took 2 min 47 s on a MacBook Pro with an M4 Pro, once the image was in.
          
+            
          $ nextflow run PathoGenOmics-Lab/AMAP -profile docker,test \
+    --skip_functional_annotation --outdir amap-test
          
+            
          If the whole chain works, it ends like this:
          
+            
          =======================================
+โœ“ COMPLETED
+=======================================
+Pipeline    : PathoGenOmics-Lab/AMAP v1.0.1
+Nextflow    : 26.10.2
+Profile     : docker,test
+Duration    : 2m 47s
+Succeeded   : 10
+Failed      : 0
+Ignored     : 0
+Output      : amap-test
+Work dir    : /Users/you/amap-check/work
+=======================================
          
+            
          The first run pulls the image: about 3.4 GB compressed, 12 GB on disk, once.
          
           
          
         
          3. 
         
        3. 
           
           
          
             
          Run your genomes
          
-            
          A sample sheet with two columns and a real tab between them, a check that takes seconds, then the run. Pick what you have and copy the command.
          
-            
          
-              
          samples.tsv
          sampleId	fasta
-CDC1551	/data/cdc1551.fna
-Erdman	/data/erdman.fna
          
-              
          Check it first
          $ nextflow run PathoGenOmics-Lab/AMAP \
-    -profile docker \
-    --tsv samples.tsv --validate_only
          
+            
          A sample sheet with two columns and a real tab between them, then two commands: a check that takes seconds, and the run. Pick what you have and copy them.
          
+            
          
+              
          samples.tsv
          sampleId	fasta
+CDC1551	/data/cdc1551.fna
+Erdman	/data/erdman.fna
          
+              
          Paths are absolute, or relative to where you launch Nextflow. Sample IDs take letters, digits and underscores. Preparing inputs.
          
             
          
             
          
-              
          
+              
          
                 
          
                   
          Where will it run?
          
-                  
          
-                    
-                    
-                    
+                  
          
+                    
+                    
+                    
                   
          
                 
          
                 
          
                   
          Which genomes?
          
-                  
          
-                    
-                    
+                  
          
+                    
+                    
                   
          
                 
          
                 
          
                   
          Anything else?
          
-                  
+                  
                   
                   
                 
          
               
          
-              
          
-                
          
-                
            
-                
+              
            
+                
            Check it first: the same command, with --validate_only
            
+                
            $ nextflow run PathoGenOmics-Lab/AMAP \
+    -profile docker \
+    --tsv samples.tsv \
+    --validate_only
            
+                
            Seconds per genome. A check that fails stops and prints its whole report. One that passes prints nothing, warnings included: its verdict is in results/pipeline_info/validation.log, here for the tutorial's genome:
            
+                
            Results: 1 OK, 0 warnings, 0 failed
+Total: 0 error(s), 0 warning(s)
            
+                
            Then run it
            
+                
            $ nextflow run PathoGenOmics-Lab/AMAP \
+    -profile docker \
+    --tsv samples.tsv
            
+                
            • -profileEvery tool runs inside the AMAP image, through Docker.
            • --tsvYour sample sheet: sampleId and fasta, tab separated.
            
+                
               
            
             
            
-            
            First time? The tutorial runs one real genome from an empty folder, with the numbers to check yours against. Every option is in Parameters.
            
+            
            First time? The tutorial runs one real genome, with numbers to check yours against. Every option is in Parameters.
            
+          
            
+        
            • 
+        
          • 
+          
+          
            
+            
            Open what came out
            
+            
            Open results/amap_report.html first: every run writes it (the test of step 2, in amap-test/), and it answers the first questions genome by genome. A full run, with eggNOG, also writes results/final_report.html, the whole run on one page. The corrected files are listed under What you get.
            
           
            
         
            • 
       
          
@@ -163,50 +232,56 @@ 
          Run your genomes
          
       
          
         
          What AMAP decides
          
         
          Each call, held up against the reference
          
-        
          Every Bakta call is compared with where the reference proteins align, on the same strand of the same contig. That puts it in one of four cases, and a reference gene with no call at all is a fifth. These are real genes of CDC1551: point at one for its coordinates.
          
+        
          Every Bakta call is compared with where the reference proteins align, on its strand and contig. That puts it in one of four cases, numbered as the code numbers them: 2 and 3 agree, 4 conflicts, 1 has no reference. A reference gene with no call at all is a fifth. These are real genes of CDC1551: point attap one for its coordinates.
          
       
          
+      
          Five outcomes, on real genes of CDC1551
          
+      
          Five cards: swipe for the next.
          
       
          
         
          
-          
          Case 2
          Perfect match
          
+          
          Case 2
          Perfect match
          
           
          Same start, same end, same strand and the same length as the aligned protein.
          
           
          
           
          AMAP keepsBakta's call, named after the protein: Rv0002, dnaN.
          
         
          
         
          
-          
          Case 3
          Imperfect match
          
-          
          The same coordinates, but the alignment's length differs: H37Rv's fhaA has 6 residues that CDC1551's lacks.
          
+          
          Case 3
          Imperfect match
          
+          
          The same coordinates, but the alignment's length differs: H37Rv's fhaA has 6 residues that CDC1551's lacks.
          
           
          
           
          AMAP keepsBakta's call, named Rv0020c.
          
         
          
         
          
-          
          Case 4
          Conflicting overlap
          
+          
          Case 4
          Conflicting overlap
          
           
          
             
             
             
           
          
-          
          Bakta's dnaA starts 33 bp after the aligned protein does. Where the two disagree, the reference wins by default.
          
-          
-          
+          
          
+            
          Bakta's dnaA starts 33 bp after the aligned protein does. Where the two disagree, the reference wins by default.
          
+            
+            
+          
          
           
          
-          
          AMAP keepsthe reference's coordinates, 1 to 1,524.
          
+          
          AMAP keepsthe reference's coordinates, 1 to 1,524.
          
         
          
         
          
-          
          Case 1
          Potentially novel
          
+          
          Case 1
          Potentially novel
          
           
          
             
             
           
          
-          
          Nothing from H37Rv aligns to these four calls: a stretch CDC1551 has and H37Rv lacks. None is a call Bakta is known to make wrongly, and none matches the H37Rv genome at 80% identity.
          
-          
+          
          
+            
          Nothing from H37Rv aligns to these four calls: a stretch CDC1551 has and H37Rv lacks. None is a call Bakta is known to make wrongly, and none matches the H37Rv genome at 80% identity.
          
+            
+          
          
           
          
-          
          AMAP keepsall four, under Bakta's locus tags.
          
+          
          AMAP keepsall four, under Bakta's locus tags.
          
         
          
         
          
           
          Missing
          A reference gene Bakta missed
          
           
          H37Rv has a 71-residue protein, Rv0031, between two of Bakta's calls, and Bakta called nothing there. No gene accounts for the alignment.
          
           
          
-          
          AMAP keepsthe gene, put back at the alignment: 33,526 to 33,738.
          
+          
          AMAP keepsthe gene, put back at the alignment: 33,526 to 33,738.
          
         
          
       
          
       
          Then come the harder ones: a protein placed on its paralog's locus, proteins too small for miniprot, genes a contig end cuts, frameshifts and lost stops, and the real copies of a gene. How it works, step by step.
          
@@ -218,19 +293,21 @@ 
          Each call, held up against the reference
          
       
          
         
          Does it work?
          
         
          Checked against genomes whose answer is known
          
-        
          The accuracy benchmark runs the correction on seven genomes and scores it against an annotation AMAP did not make: the reference's own genes, carried onto the other genomes by a whole-genome alignment. Every gene a genome carries unchanged is found at exactly its coordinates, under its name. Of the genes that changed, which have no exact answer, 99.8 to 100% still sit where the alignment puts them, under their name.
          
+        
          Seven genomes, scored against genes AMAP did not annotate: the reference's own, carried onto each genome by a whole-genome alignment. AMAP finds every unchanged gene at exactly its coordinates, under its name; Bakta alone, scored the same way, finds 67 to 90% of them.
          
+        
          100%of unchanged genes found at exactly their coordinates, in all seven genomes. Bakta alone: 67 to 90%.
          
       
          
-      
          
-        
          Unchanged genes, at exactly their coordinatesChanged genes, found under their name
          
-
          M. tuberculosis H37Rvagainst itself4,026 / 4,026no gene changed
          
-
          H37Rv, another assemblyCP110619, against H37Rv3,986 / 3,98650 / 50
          
-
          M. tuberculosis CDC1551against H37Rv3,097 / 3,097860 / 862
          
-
          M. bovis AF2122/97against H37Rv2,391 / 2,3911,515 / 1,515
          
-
          CDC1551 in 40 contigsa draft, against H37Rv3,072 / 3,072no gene changed
          
-
          E. coli K-12 MG1655against itself4,286 / 4,286no gene changed
          
-
          E. coli O157:H7 Sakaiagainst K-12220 / 2203,363 / 3,367
          
+      
          Genes found per genome, by AMAP and by Bakta alone
          
+      
          
+        
          Unchanged genes, at exactly their coordinatesChanged genes, found under their name
          
+
          M. tuberculosis H37Rvagainst itself4,026 / 4,026Bakta alone: 2,905no gene changed
          
+
          H37Rv, another assemblyCP110619, against H37Rv3,986 / 3,986Bakta alone: 2,88050 / 50Bakta alone: 29
          
+
          M. tuberculosis CDC1551against H37Rv3,097 / 3,097Bakta alone: 2,245860 / 862Bakta alone: 705
          
+
          M. bovis AF2122/97against H37Rv2,391 / 2,391Bakta alone: 1,7421,515 / 1,515Bakta alone: 1,290
          
+
          CDC1551 in 40 contigsa draft, against H37Rv3,072 / 3,072Bakta alone: 2,231no gene changed
          
+
          E. coli K-12 MG1655against itself4,286 / 4,286Bakta alone: 3,841no gene changed
          
+
          E. coli O157:H7 Sakaiagainst K-12220 / 220Bakta alone: 1483,363 / 3,367Bakta alone: 3,202
          
       
          
-      
          Scores of tests/benchmark, which anyone can rerun in the container; the truth comes from the references and minimap2, never from AMAP.
          
+      
          Each bar is AMAP; shaded up to the line, what Bakta finds alone. Scores of tests/benchmark, which anyone can rerun in the container; the truth comes from the references and minimap2, never from AMAP.
          
     
          
   
          
 
@@ -239,13 +316,14 @@ 
          Checked against genomes whose answer is known
          
       
          
         
          The pipeline
          
         
          Where each step happens
          
-        
          Stations are the processes of main.nf; lines are what travels between them. Pick a station, or flip an option to see what a run leaves out.
          
+        
          Stations are the processes of main.nf; lines are what travels between them. Pick a station, or flip an option to see what a run leaves out.
          
       
          
+      
          Every process of main.nf, in the order it runs them
          
       
          
         
          
           Run with
-          
-          
+          
+          
           
         
          
         
@@ -425,7 +541,7 @@ 
          Open source, on GitHub
          
   
          
     
          
       
          Cite
          
-      
          Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software].
          
+      
          Ruiz-Rodriguez, P., & Coscolla, M. (2026). AMAP (Version 1.0.1) [Computer software].
          
       
          Software: doi:10.5281/zenodo.8024446
          
     
          
     
          
@@ -435,7 +551,7 @@ 
          Learn more
          
         
        4. Documentation
          5. 
         
        5. How it works
          6. 
         
        6. Parameters
          7. 
-        
        7. Changelog
          8. 
+        
        8. Changelog
          9. 
       
     
          
     
          
@@ -444,10 +560,10 @@ 
          Project
          
         
        9. GitHub
          10. 
         
        10. Report a bug
          11. 
         
        11. Docker image
          12. 
-        
        12. GPL-3.0 · PathoGenOmics Lab
          13. 
       
     
          
   
          
+  
          PathoGenOmics LabGPL-3.0AMAP 1.0.1
          
 
 
 
diff --git a/docs/user-guide/amap-report.html b/docs/user-guide/amap-report.html
index b6e1a1b..c3c201d 100644
--- a/docs/user-guide/amap-report.html
+++ b/docs/user-guide/amap-report.html
@@ -3,145 +3,170 @@
 
 
 
-The interactive report | AMAP docs
-
+The report by question | AMAP docs
+
 
+
 
 
-
-
+
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
          
   
          
     
-      AMAP
+      AMAP
     
     
   
          
 
          
 
          
-
+  
+  
+
          
 
          
-
          Docs / User guide
          
-
          The interactive report
          
-
          Next to final_report.html, every run now writes results/amap_report.html: one file that answers, section by section, the questions a run raises. Did each genome annotate well? Where on the reference are genes missing or broken? What happened to one gene in every genome? What did the correction change? Every table filters, sorts and saves as CSV, every figure saves as SVG, and a click on a genome or a gene opens it.
          
-
          It is written with or without the functional annotation. It needs only what the correction leaves, so a run with --skip_functional_annotation, which gets no final_report.html, still gets this one; the eggNOG sections then say that the run skipped them.
          
-
          It is self-contained: the data travels inside the file, compressed, and nothing is fetched from anywhere, so it opens with no network and survives being emailed. Fifteen MTBC genomes make a file of about half a megabyte, against 4.4 MB for final_report.html with twelve. It is dark, like the website, with a switch to light at the top right.
          
-
          The sections#
          
-
          1. Did each genome annotate well? One row per genome: lineage, genome size, contigs and N50, Bakta's calls and AMAP's genes, the share of the reference genes it has, and how many are missing, pseudogenes, partial, found more than once or outside the reference. A value far from those of comparable genomes is flagged (see How a value is flagged). A click on a genome opens it: its numbers, and its genes in each state, each list searchable and saved as CSV.
          
-
          2. Where on the reference are genes missing or broken? Every reference gene at its place on the reference, one row per genome, the genomes grouped by lineage. A missing gene fills its row; a pseudogene is a band through the middle; a second copy a band on top. Deletions that genomes share show as blocks: with MTBC genomes, the regions of difference. Drag across the map to zoom into a region; its genes are then listed with how many genomes have each in each state. Below the map, Blocks of missing genes lists every run of two or more consecutive reference genes missing from a genome, grouped where genomes overlap, and says which lineages lack each block in all their genomes.
          
-
          3. What happened to one gene in every genome? Type a locus, a gene name or part of a product. The card gives the reference's own coordinates, length, product, functional category and, from Mycobrowser, whether the reference calls it a pseudogene and which drugs it has resistance mutations for. The table gives, for every genome, its state, locus tag, contig, coordinates, strand, length against the reference, the note of a pseudogene and how AMAP got the gene.
          
-
          4. What did AMAP change in Bakta's annotation? A waterfall from Bakta's calls to AMAP's genes: the case 1 calls the filter removed, the case 4 calls the reference replaced, then the genes placed from the reference over a call and those added where Bakta called nothing. For one genome a click on a bar lists its genes; for the calls AMAP did not keep, Bakta's coordinates next to those of the gene that took their place. The correction algorithm says what each case means.
          
-
          5. Which genes are broken, and in which lineages? Every gene AMAP flags as a pseudogene in at least one genome, against the genomes, with the reason. First the genes broken in every genome of some lineages and in no other, then those broken in every genome, then those broken in all but one or two, then the rest. With Mycobrowser, each is compared with the reference's own pseudogenes.
          
-
          6. What do the genomes carry that the reference does not? How many genomes carry each reference gene, with the genes some genomes lack; the genes that name no reference gene, grouped by product across genomes; the reference genes with more than one copy; and, when the run has them, the pangenome and the homologous communities.
          
-
          7. Do the lineages differ in function? The genes each lineage loses (missing or pseudogene) by Mycobrowser category, and with the functional annotation, genes by COG category and KEGG pathway. One line per category, a dot per lineage, the categories that differ most between lineages first.
          
-
          8. How was this made? AMAP's version, the command, the reference, every parameter (pipeline_info/params.yml), every tool's version (pipeline_info/software_versions.tsv), and what each state means.
          
+
          Docs / User guide
          Describes AMAP on main, ahead of 1.0.1: what changed
          
+
          The report by question: amap_report.html
          
+
          Every question the report answers, with or without eggNOG.
          
+
          6 min read
          
+
          On this page
          
+
+
          Every run writes results/amap_report.html, next to the full report, final_report.html, when the run has one: one file that answers, section by section, the questions a run raises. Did each genome annotate well? Where on the reference are genes missing or broken? What happened to one gene in every genome? What did the correction change? Every table filters, sorts and saves as CSV, every figure saves as SVG, and a click on a genome or a gene opens it.
          
+
          It is written with or without the functional annotation. It needs only what the correction leaves, so a run with --skip_functional_annotation has no final_report.html; amap_report.html is still written, without its eggNOG sections, which say that the run skipped them.
          
+
          It is self-contained: the data travels inside the file, compressed, and nothing is fetched from anywhere, so it opens with no network and survives being emailed. Fifteen genomes of the Mycobacterium tuberculosis complex (MTBC) make a file of about half a megabyte, against 4.5ย MB for final_report.html with twelve. It is dark, like the website, with a switch to light at the top right.
          
+
          The sections
          #
          
+
          1. Did each genome annotate well?
          #
          
+
          One row per genome: lineage, genome size, contigs and N50, Bakta's calls and AMAP's genes, the share of the reference genes it has, and how many are missing, pseudogenes, partial, found more than once or outside the reference. A value far from those of comparable genomes is flagged (see How a value is flagged). A click on a genome opens it: its numbers, and its genes in each state, each list searchable and saved as CSV.
          
+
          2. Where on the reference are genes missing or broken?
          #
          
+
          Every reference gene at its place on the reference, one row per genome, the genomes grouped by lineage. A missing gene fills its row; a pseudogene is a band through the middle; a second copy a band on top. Deletions that genomes share show as blocks: with MTBC genomes, the regions of difference. Drag across the map to zoom into a region; its genes are then listed with how many genomes have each in each state. Below the map, Blocks of missing genes lists every run of two or more consecutive reference genes missing from a genome, grouped where genomes overlap, and says which lineages lack each block in all their genomes.
          
+
          3. What happened to one gene in every genome?
          #
          
+
          Type a locus, a gene name or part of a product. The card gives the reference's own coordinates, length, product, functional category and, from Mycobrowser, whether the reference calls it a pseudogene and which drugs it has resistance mutations for. The table gives, for every genome, its state, locus tag, contig, coordinates, strand, length against the reference, the note of a pseudogene and how AMAP got the gene.
          
+
          4. What did AMAP change in Bakta's annotation?
          #
          
+
          A waterfall from Bakta's calls to AMAP's genes: the caseย 1 calls the filter removed, the caseย 4 calls the reference replaced, then the genes placed from the reference over a call and those added where Bakta called nothing. For one genome a click on a bar lists its genes; for the calls AMAP did not keep, Bakta's coordinates next to those of the gene that took their place. The correction algorithm says what each case means.
          
+
          5. Which genes are broken, and in which lineages?
          #
          
+
          Every gene AMAP flags as a pseudogene in at least one genome, against the genomes, with the reason. First the genes broken in every genome of some lineages and in no other, then those broken in every genome, then those broken in all but one or two, then the rest. With Mycobrowser, each is compared with the reference's own pseudogenes.
          
+
          6. What do the genomes carry that the reference does not?
          #
          
+
          How many genomes carry each reference gene, with the genes some genomes lack; the genes that name no reference gene, grouped by product across genomes; the reference genes with more than one copy; and, when the run has them, the pangenome and the homologous communities.
          
+
          7. Do the lineages differ in function?
          #
          
+
          The genes each lineage loses (missing or pseudogene) by Mycobrowser category, and with the functional annotation, genes by COG category and KEGG pathway. One line per category, a dot per lineage, the categories that differ most between lineages first.
          
+
          8. How was this made?
          #
          
+
          AMAP's version, the command, the reference, every parameter (pipeline_info/params.yml), every tool's version (pipeline_info/software_versions.tsv), and what each state means.
          
 
          The bar under the top one filters every section at once: a lineage's pill shows or hides it (Alt or Cmd and a click keeps that lineage alone), and the box keeps the genomes whose name contains what is typed.
          
-
          Gene states#
          
-
          Each reference gene gets one state in each genome, from the genes that name it in the genome's final annotation, AMAP/<name>/<name>.AMAP.tsv.
          
-
          StateWhat it means
          IntactA gene names it, and nothing below applies
          MissingNo gene names it
          PseudogeneThe best gene naming it is flagged as a pseudogene: an internal stop, a lost stop or a frame that runs on past the reference protein
          PartialThe best gene naming it runs into a contig end and is cut there
          More copiesMore than one gene names it alone
          Length differsOne intact copy, of another length than the reference gene
          Shared copyOnly a gene that names several reference genes at once names it, such as an IS6110 copy that matches all seventeen; counted as found
          
+
          Gene states
          #
          
+
          Each reference gene gets one state in each genome, from the genes that name it in the genome's final annotation, AMAP/{lineage}_{sampleId}/{lineage}_{sampleId}.AMAP.tsv.
          
+
          StateWhat it means
          IntactA gene names it, and nothing below applies
          MissingNo gene names it
          PseudogeneThe best gene naming it is flagged as a pseudogene: an internal stop, a lost stop or a frame that runs on past the reference protein
          PartialThe best gene naming it runs into a contig end and is cut there
          More copiesMore than one gene names it alone
          Length differsOne intact copy, of another length than the reference gene
          Shared copyOnly a gene that names several reference genes at once names it, such as an IS6110 copy that matches all seventeen; counted as found
          
 
          When several apply, the first in the table wins: a pseudogene that is also cut by a contig end is a pseudogene. "Reference genes found" counts every state but missing.
          
-
          How a value is flagged#
          
+
          How aย value is flagged
          #
          
 
          Each genome is compared with its own lineage when that lineage has five genomes or more, and with every other genome otherwise. A value is flagged when it lies more than 3.5 robust standard deviations (1.4826 times the median absolute deviation) from their median, on the side that is bad: more missing genes, pseudogenes, partial genes, contigs or genes outside the reference, fewer reference genes found, a smaller N50, a genome size either way. Each also has to be further from the median than a minimum (five genes, five contigs, half a point of reference genes found, 1% of the genome size or 5% of the N50), so that identical genomes do not flag a gene or two of difference. Fewer than four genomes are not compared at all.
          
-
          A flag says a genome is unusual, not that it is wrong. M. bovis among M. tuberculosis genomes is flagged for its missing genes, which are its regions of difference. A genome with twice the pseudogenes of the others is worth a look at its assembly: frameshifts from sequencing errors look like pseudogenes.
          
-
          What each section needs#
          
-
          InputFromWithout it
          <name>.AMAP.tsvFIX_CDSNo report
          The reference proteomethe referenceNo report
          Case summaries and tablesTYPE_CASESNo lineages, no correction section
          Bakta's folderBAKTA_ANNOTATORNo genome sizes, contigs or products of Bakta's calls
          Mycobrowser tablethe reference, MTBC onlyNames and products from the genomes' own annotation; no categories or drug resistance
          Master tablesGET_OUTPUTS, functional annotationNo COG categories or KEGG pathways
          Pangenome matrixREPORT_INPUTS, functional annotationNo pangenome
          Homologous communitiesGET_ALLDUP, functional annotationNo communities
          params.yml, software_versions.tsvDUMP_PARAMS, SOFTWARE_VERSIONSNo parameters or versions
          
-
          A section without its inputs says which and why instead of drawing anything. With a custom reference, under --skip_typing and --skip_mycobrowser, every genome is Untyped and the names and products come from Bakta and eggNOG.
          
-
          Genomes of the sample sheet that produced no annotation, because the pipeline ignored a task that kept failing, are named at the top of the report.
          
-
          Size#
          
-
          The data of each genome is a few kilobytes: its genes are stored as how far each lies from where the previous gene and the reference predict it, which for a genome collinear with its reference is mostly nothing. Up to 100 genomes, the report also lists every Bakta call AMAP did not keep, with its coordinates; past that it only counts them, to keep the file small, and CDS_CASES/<name>/ has them for any run.
          
-
          GenomesReport
          150.5 MB, measured
          1001.5 to 1.7 MB
          3001.9 to 2.1 MB
          5002.9 to 3.3 MB
          
+
          A flag says a genome is unusual, not that it is wrong. M.ย bovis among M.ย tuberculosis genomes is flagged for its missing genes, which are its regions of difference. A genome with twice the pseudogenes of the others is worth a look at its assembly: frameshifts from sequencing errors look like pseudogenes.
          
+
          What each section needs
          #
          
+
          InputFromWithout it
          *.AMAP.tsvFIX_CDSNo report
          The reference proteomethe referenceNo report
          Case summaries and tablesTYPE_CASESNo lineages, no correction section
          Bakta's folderBAKTA_ANNOTATORNo genome sizes, contigs or products of Bakta's calls
          Mycobrowser tablethe reference, MTBC onlyNames and products from the genomes' own annotation; no categories or drug resistance
          Master tablesGET_OUTPUTS, functional annotationNo COG categories or KEGG pathways
          Pangenome matrixREPORT_INPUTS, functional annotationNo pangenome
          Homologous communitiesGET_ALLDUP, functional annotationNo communities
          params.yml, software_versions.tsvDUMP_PARAMS, SOFTWARE_VERSIONSNo parameters or versions
          
+
          A section without its inputs says which and why instead of drawing anything. With a custom reference, under --skip_typing and --skip_mycobrowser, every genome is Untyped and the names and products come from Bakta and eggNOG.
          
+
          A genome that was classified but has no corrected genes, because the pipeline set aside its correction after it kept failing, is named at the top of the report. One whose Bakta, miniprot or classification task was set aside never reaches the report: see a sample is missing.
          
+
          Size
          #
          
+
          The data of each genome is a few kilobytes: its genes are stored as how far each lies from where the previous gene and the reference predict it, which for a genome collinear with its reference is mostly nothing. Up to 100 genomes, the report also lists every Bakta call AMAP did not keep, with its coordinates; past that it only counts them, to keep the file small, and CDS_CASES/{lineage}_{sampleId}/ has them for any run.
          
+
          GenomesReport
          150.5ย MB, measured
          1001.5ย toย 1.7ย MB
          3001.9ย toย 2.1ย MB
          5002.9ย toย 3.3ย MB
          
 
          The sizes past fifteen are worked out from the fifteen MTBC genomes of eight lineages the report was measured on, the larger figure with each genome's data compressed on its own and the smaller with the sharing measured between those fifteen. In the browser, five hundred genomes open in about three seconds.
          
-
          Writing one by hand#
          
-
          bin/report/amap_report.py needs only Python 3.9 or later. Pointed at a results/ folder it reads the files where the pipeline publishes them:
          
-
          python3 bin/report/amap_report.py \
-    --amap results/AMAP \
-    --cases results/CDS_CASES \
-    --bakta results/BAKTA \
-    --master results/AMAP \
-    --ref-proteome ref_proteome.faa \
-    --mycobrowser Mycobacterium_tuberculosis_H37Rv_txt_v4.txt \
-    --versions results/pipeline_info/software_versions.tsv \
-    --params results/pipeline_info/params.yml \
-    --out amap_report.html
          
-
          The reference proteome is the one the run used: for H37Rv, the AMAP container has it at /reference/aa_H37Rv.translated.fas, and the Mycobrowser table at /reference/Mycobacterium_tuberculosis_H37Rv_txt_v4.txt.
          
-
          Note
          The page reads its data with the browser's own decompression, which every current browser has: Chrome and Edge 80, Firefox 113 and Safari 16.4, and later. An older one says so instead of drawing the report.
          
-
          Edit this page on GitHub
          
-
+
          Writing one by hand
          #
          
+
          bin/report/amap_report.py needs only Python 3.9 or later. Pointed at a results/ folder it reads the files where the pipeline publishes them:
          
+
          Terminal
          $ python3 bin/report/amap_report.py \
+    --amap results/AMAP \
+    --cases results/CDS_CASES \
+    --bakta results/BAKTA \
+    --master results/AMAP \
+    --ref-proteome ref_proteome.faa \
+    --mycobrowser Mycobacterium_tuberculosis_H37Rv_txt_v4.txt \
+    --versions results/pipeline_info/software_versions.tsv \
+    --params results/pipeline_info/params.yml \
+    --out amap_report.html
          
+
          The reference proteome is the one the run used: for H37Rv, the AMAP container has it at /reference/aa_H37Rv.translated.fas, and the Mycobrowser table at /reference/Mycobacterium_tuberculosis_H37Rv_txt_v4.txt.
          
+
          Note
          The page reads its data with the browser's own decompression, which every current browser has: Chrome and Edge 80, Firefox 113 and Safari 16.4, and later. An older one says so instead of drawing the report.
          
+
          Back to the top
          
+
          Edit this page on GitHub
          
+
 
          
-
 
          
 
-
 
 
 
diff --git a/docs/user-guide/amap-report.md b/docs/user-guide/amap-report.md
index 54e12ac..04d5acc 100644
--- a/docs/user-guide/amap-report.md
+++ b/docs/user-guide/amap-report.md
@@ -1,77 +1,90 @@
-# The interactive report
+# The report by question: amap_report.html
 
-Next to `final_report.html`, every run now writes `results/amap_report.html`:
-one file that answers, section by section, the questions a run raises. Did each
+Every run writes `results/amap_report.html`, next to
+[the full report](report.md), `final_report.html`, when the run has one: one
+file that answers, section by section, the questions a run raises. Did each
 genome annotate well? Where on the reference are genes missing or broken? What
 happened to one gene in every genome? What did the correction change? Every
 table filters, sorts and saves as CSV, every figure saves as SVG, and a click on
 a genome or a gene opens it.
 
 It is written with or without the functional annotation. It needs only what the
-correction leaves, so a run with `--skip_functional_annotation`, which gets no
-`final_report.html`, still gets this one; the eggNOG sections then say that the
-run skipped them.
+correction leaves, so a run with `--skip_functional_annotation` has no
+`final_report.html`; `amap_report.html` is still written, without its eggNOG
+sections, which say that the run skipped them.
 
 It is self-contained: the data travels inside the file, compressed, and nothing
 is fetched from anywhere, so it opens with no network and survives being
-emailed. Fifteen MTBC genomes make a file of about half a megabyte, against
-4.4 MB for `final_report.html` with twelve. It is dark, like the website, with a
+emailed. Fifteen genomes of the *Mycobacterium tuberculosis* complex (MTBC) make a file of about half a megabyte, against
+4.5 MB for `final_report.html` with twelve. It is dark, like the website, with a
 switch to light at the top right.
 
 ## The sections
 
-**1. Did each genome annotate well?** One row per genome: lineage, genome size,
-contigs and N50, Bakta's calls and AMAP's genes, the share of the reference
-genes it has, and how many are missing, pseudogenes, partial, found more than
-once or outside the reference. A value far from those of comparable genomes is
-flagged (see [How a value is flagged](#how-a-value-is-flagged)). A click on a
-genome opens it: its numbers, and its genes in each state, each list searchable
-and saved as CSV.
-
-**2. Where on the reference are genes missing or broken?** Every reference gene
-at its place on the reference, one row per genome, the genomes grouped by
-lineage. A missing gene fills its row; a pseudogene is a band through the middle;
-a second copy a band on top. Deletions that genomes share show as blocks: with
-MTBC genomes, the regions of difference. Drag across the map to zoom into a
-region; its genes are then listed with how many genomes have each in each
-state. Below the map, **Blocks of missing genes** lists every run of two or
-more consecutive reference genes missing from a genome, grouped where genomes
-overlap, and says which lineages lack each block in all their genomes.
-
-**3. What happened to one gene in every genome?** Type a locus, a gene name or
-part of a product. The card gives the reference's own coordinates, length,
-product, functional category and, from Mycobrowser, whether the reference calls
-it a pseudogene and which drugs it has resistance mutations for. The table
-gives, for every genome, its state, locus tag, contig, coordinates, strand,
-length against the reference, the note of a pseudogene and how AMAP got the gene.
-
-**4. What did AMAP change in Bakta's annotation?** A waterfall from Bakta's
-calls to AMAP's genes: the case 1 calls the filter removed, the case 4 calls the
-reference replaced, then the genes placed from the reference over a call and
-those added where Bakta called nothing. For one genome a click on a bar lists
-its genes; for the calls AMAP did not keep, Bakta's coordinates next to those of
-the gene that took their place. [The correction algorithm](../how-it-works/correction-algorithm.md)
-says what each case means.
-
-**5. Which genes are broken, and in which lineages?** Every gene AMAP flags as a
-pseudogene in at least one genome, against the genomes, with the reason. First
-the genes broken in every genome of some lineages and in no other, then those
-broken in every genome, then those broken in all but one or two, then the rest.
-With Mycobrowser, each is compared with the reference's own pseudogenes.
-
-**6. What do the genomes carry that the reference does not?** How many genomes
-carry each reference gene, with the genes some genomes lack; the genes that name
-no reference gene, grouped by product across genomes; the reference genes with
-more than one copy; and, when the run has them, the pangenome and the
-homologous communities.
-
-**7. Do the lineages differ in function?** The genes each lineage loses
-(missing or pseudogene) by Mycobrowser category, and with the functional
-annotation, genes by COG category and KEGG pathway. One line per category, a dot
-per lineage, the categories that differ most between lineages first.
-
-**8. How was this made?** AMAP's version, the command, the reference, every
-parameter (`pipeline_info/params.yml`), every tool's version
+### 1. Did each genome annotate well?
+
+One row per genome: lineage, genome size, contigs and N50, Bakta's calls and
+AMAP's genes, the share of the reference genes it has, and how many are missing,
+pseudogenes, partial, found more than once or outside the reference. A value far
+from those of comparable genomes is flagged (see [How a value is
+flagged](#how-a-value-is-flagged)). A click on a genome opens it: its numbers,
+and its genes in each state, each list searchable and saved as CSV.
+
+### 2. Where on the reference are genes missing or broken?
+
+Every reference gene at its place on the reference, one row per genome, the
+genomes grouped by lineage. A missing gene fills its row; a pseudogene is a band
+through the middle; a second copy a band on top. Deletions that genomes share
+show as blocks: with MTBC genomes, the regions of difference. Drag across the
+map to zoom into a region; its genes are then listed with how many genomes have
+each in each state. Below the map, **Blocks of missing genes** lists every run
+of two or more consecutive reference genes missing from a genome, grouped where
+genomes overlap, and says which lineages lack each block in all their genomes.
+
+### 3. What happened to one gene in every genome?
+
+Type a locus, a gene name or part of a product. The card gives the reference's
+own coordinates, length, product, functional category and, from Mycobrowser,
+whether the reference calls it a pseudogene and which drugs it has resistance
+mutations for. The table gives, for every genome, its state, locus tag, contig,
+coordinates, strand, length against the reference, the note of a pseudogene and
+how AMAP got the gene.
+
+### 4. What did AMAP change in Bakta's annotation?
+
+A waterfall from Bakta's calls to AMAP's genes: the case 1 calls the filter
+removed, the case 4 calls the reference replaced, then the genes placed from the
+reference over a call and those added where Bakta called nothing. For one genome
+a click on a bar lists its genes; for the calls AMAP did not keep, Bakta's
+coordinates next to those of the gene that took their place. [The correction
+algorithm](../how-it-works/correction-algorithm.md) says what each case means.
+
+### 5. Which genes are broken, and in which lineages?
+
+Every gene AMAP flags as a pseudogene in at least one genome, against the
+genomes, with the reason. First the genes broken in every genome of some
+lineages and in no other, then those broken in every genome, then those broken
+in all but one or two, then the rest. With Mycobrowser, each is compared with
+the reference's own pseudogenes.
+
+### 6. What do the genomes carry that the reference does not?
+
+How many genomes carry each reference gene, with the genes some genomes lack;
+the genes that name no reference gene, grouped by product across genomes; the
+reference genes with more than one copy; and, when the run has them, the
+pangenome and the homologous communities.
+
+### 7. Do the lineages differ in function?
+
+The genes each lineage loses (missing or pseudogene) by Mycobrowser category,
+and with the functional annotation, genes by COG category and KEGG pathway. One
+line per category, a dot per lineage, the categories that differ most between
+lineages first.
+
+### 8. How was this made?
+
+AMAP's version, the command, the reference, every parameter
+(`pipeline_info/params.yml`), every tool's version
 (`pipeline_info/software_versions.tsv`), and what each state means.
 
 The bar under the top one filters every section at once: a lineage's pill shows
@@ -81,7 +94,7 @@ the genomes whose name contains what is typed.
 ## Gene states
 
 Each reference gene gets one state in each genome, from the genes that name it
-in the genome's final annotation, `AMAP//.AMAP.tsv`.
+in the genome's final annotation, `AMAP/{lineage}_{sampleId}/{lineage}_{sampleId}.AMAP.tsv`.
 
 | State | What it means |
 | :--- | :--- |
@@ -120,7 +133,7 @@ pseudogenes.
 
 | Input | From | Without it |
 | :--- | :--- | :--- |
-| `.AMAP.tsv` | `FIX_CDS` | No report |
+| `*.AMAP.tsv` | `FIX_CDS` | No report |
 | The reference proteome | the reference | No report |
 | Case summaries and tables | `TYPE_CASES` | No lineages, no correction section |
 | Bakta's folder | `BAKTA_ANNOTATOR` | No genome sizes, contigs or products of Bakta's calls |
@@ -134,8 +147,10 @@ A section without its inputs says which and why instead of drawing anything.
 With a custom reference, under `--skip_typing` and `--skip_mycobrowser`, every
 genome is `Untyped` and the names and products come from Bakta and eggNOG.
 
-Genomes of the sample sheet that produced no annotation, because the pipeline
-ignored a task that kept failing, are named at the top of the report.
+A genome that was classified but has no corrected genes, because the pipeline
+set aside its correction after it kept failing, is named at the top of the
+report. One whose Bakta, miniprot or classification task was set aside never
+reaches the report: see [a sample is missing](../deployment/troubleshooting.md#a-sample-is-missing-from-the-results-ignored--1).
 
 ## Size
 
@@ -143,7 +158,7 @@ The data of each genome is a few kilobytes: its genes are stored as how far each
 lies from where the previous gene and the reference predict it, which for a
 genome collinear with its reference is mostly nothing. Up to 100 genomes, the
 report also lists every Bakta call AMAP did not keep, with its coordinates; past
-that it only counts them, to keep the file small, and `CDS_CASES//` has
+that it only counts them, to keep the file small, and `CDS_CASES/{lineage}_{sampleId}/` has
 them for any run.
 
 | Genomes | Report |
diff --git a/docs/user-guide/custom-reference.html b/docs/user-guide/custom-reference.html
index 4062578..98c16ff 100644
--- a/docs/user-guide/custom-reference.html
+++ b/docs/user-guide/custom-reference.html
@@ -4,119 +4,139 @@
 
 
 Custom reference | AMAP docs
-
+
 
+
 
 
 
-
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
          
   
          
     
-      AMAP
+      AMAP
     
     
   
          
 
          
 
          
-
+  
+  
+
          
 
          
-
          Docs / User guide
          
-
          Using your own reference
          
-
          AMAP corrects an annotation by refereeing it against a trusted one. For MTBC that trusted annotation is H37Rv, bundled in the image. Point --ref_annotation and --ref_genome at your own and the same machinery works for any prokaryote.
          
-
          nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv \
-    --skip_typing --skip_mycobrowser \
-    --genus Escherichia --species "Escherichia coli" \
-    --ref_annotation my_reference.gbk \
-    --ref_genome my_reference.fasta
          
+
          Docs / User guide
          Describes AMAP on main, ahead of 1.0.1: what changed
          
+
          Custom reference
          
+
          Any organism, against a trusted annotation of your own.
          
+
          4 min read
          
+
          On this page
          
+
+
          AMAP corrects an annotation by refereeing it against a trusted one. For the Mycobacterium tuberculosis complex (MTBC) that trusted annotation is H37Rv, bundled in the image. Point --ref_annotation and --ref_genome at your own and the same machinery works for any prokaryote.
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv \
+    --skip_typing --skip_mycobrowser \
+    --genus Escherichia --species "Escherichia coli" \
+    --ref_annotation my_reference.gbk \
+    --ref_genome my_reference.fasta
          
 
          Both must be given together. Setting one without the other stops the run immediately.
          
-
          What counts as a good reference#
          
+
          What counts as aย good reference
          #
          
 
          A closed, well-curated genome of a close relative, ideally the type strain or a reference assembly from RefSeq. The correction assumes the reference genes are right, so its quality is the ceiling on your result.
          
-
          • --ref_annotation: GFF3 (.gff, .gff3) or GenBank (.gbk, .gb, .genbank, .gbff). Detected from the extension, with GenBank tried first for anything else.
          • --ref_genome: FASTA of the same genome. It must be the same assembly the annotation describes, since coordinates are taken at face value.
          
-
          What AMAP does with it#
          
-
          PREPARE_REFERENCE runs once, before any sample is processed, and does three things.
          
-
          --ref_annotation + --ref_genome
          1Extract the proteome
          Every CDS of your annotation becomes a protein named ID_START-END.
          4,318 proteins
          2Annotate the reference with Bakta
          The same Bakta that will annotate your samples, on a genome whose answer is known.
          4,296 Bakta calls
          3Keep what Bakta invents
          Every Bakta call that overlaps no curated gene of its own sequence: the calls it is prone to make on your samples too.
          142 false positives
          The proteome is aligned onto every sample; the false positives rule out case 1 calls.
          What happens to E. coli K-12 MG1655 given as the reference, once, before any sample is touched.
          
-
          1. Extracts the reference proteome. Every CDS becomes a protein, headed GENE_ID_START-END contig=SEQUENCE, SEQUENCE being the name --ref_genome gives the sequence the gene is on. Those embedded coordinates are not decoration: the paralog step reads them back to predict where each protein belongs in the sample, which tells its own locus from a paralog's, so the header format matters. A CDS across the origin of a circular sequence ends past the sequence's last base, the way GFF3 writes it: tagA on plasmid pO157 (92721 bp) of E. coli O157:H7 Sakai, join(92527..92721,1..2502) in its GenBank record, is pO157p01_92527-95223. From GenBank the existing /translation qualifier is used when present, otherwise the sequence is extracted and translated. From GFF3 the lines of each CDS are joined first, as GenBank's join() is, and translated with the bacterial code, reading transl_except (a selenocysteine) where it is given; on E. coli K-12 the two formats give the same 4318 proteins, and on Sakai the same 5155. A GFF3 protein under 8 amino acids is left out.
          
-
          The gene ID comes from locus_tag, then gene, then protein_id for GenBank; locus_tag, ID, gene, then Name for GFF3. A CDS with none of these is skipped.
          
-
          2. Annotates the reference genome with Bakta. The same Bakta that will annotate your samples, run on a genome whose true annotation is known, keeping the FASTA's names for its sequences (--keep-contig-headers).
          
-
          3. Works out what Bakta invents. Any Bakta CDS that does not overlap a curated gene of its own sequence is a false positive on this genome, and by extension a call Bakta is prone to make on your samples too. These become the bad_bakta set used to filter novel-gene candidates in case 1. Bakta writes a call across the origin of a sequence whose FASTA description says complete from its start to its stop, start after end (stcE on pO157 is 92563..2502), and it is compared with the genes on both sides of the origin. The calls on a sequence the annotation has no gene on are not judged: the annotation says nothing about them.
          
+
          • --ref_annotation: GFF3 (.gff, .gff3) or GenBank (.gbk, .gb, .genbank, .gbff). Detected from the extension, with GenBank tried first for anything else.
          • --ref_genome: FASTA of the same genome. It must be the same assembly the annotation describes, since coordinates are taken at face value.
          
+
          What AMAP does with it
          #
          
+
          PREPARE_REFERENCE runs once, before any sample is processed, and does three things.
          
+
          --ref_annotation + --ref_genome
          1Extract the proteome
          Every CDS of your annotation becomes a protein named ID_START-END.
          4,318 proteins
          2Annotate the reference with Bakta
          The same Bakta that will annotate your samples, on a genome whose answer is known.
          4,296 Bakta calls
          3Keep what Bakta invents
          Every Bakta call that overlaps no curated gene of its own sequence: the calls it is prone to make on your samples too.
          142 false positives
          The proteome is aligned onto every sample; the false positives rule out caseย 1 calls.
          What happens to E. coli K-12 MG1655 given as the reference, once, before any sample is touched.
          
+
          1. Extracts the reference proteome. Every CDS becomes a protein, headed GENE_ID_START-END contig=SEQUENCE, SEQUENCE being the name --ref_genome gives the sequence the gene is on.
          
+
          • The header matters. The paralog step reads its coordinates back to predict where each protein belongs in the sample, which tells its own locus from a paralog's.
          • From GenBank, the existing /translation qualifier is used when present; otherwise the sequence is extracted and translated.
          • From GFF3, the lines of each CDS are joined first, as GenBank's join() is, and translated with the bacterial code, reading transl_except (a selenocysteine) where it is given. A GFF3 protein under 8 amino acids is left out.
          • The gene ID comes from locus_tag, then gene, then protein_id for GenBank; locus_tag, ID, gene, then Name for GFF3. A CDS with none of these is skipped.
          
+
          NoteGenes across the origin
          A CDS across the origin of a circular sequence ends past the sequence's last base, the way GFF3 writes it: tagA on plasmid pO157 (92,721ย bp) of E.ย coli O157:H7 Sakai, join(92527..92721,1..2502) in its GenBank record, is pO157p01_92527-95223. On E.ย coli K-12 the two formats give the same 4,318 proteins, and on Sakai the same 5,155.
          
+
          2. Annotates the reference genome with Bakta. The same Bakta that will annotate your samples, run on a genome whose true annotation is known, keeping the FASTA's names for its sequences (--keep-contig-headers).
          
+
          3. Works out what Bakta invents. Any Bakta CDS that does not overlap a curated gene of its own sequence is a false positive on this genome, and by extension a call Bakta is prone to make on your samples too. These become the bad_bakta set used to filter novel-gene candidates in caseย 1. Bakta writes a call across the origin of a sequence whose FASTA description says complete from its start to its stop, start after end (stcE on pO157 is 92563..2502), and it is compared with the genes on both sides of the origin. The calls on a sequence the annotation has no gene on are not judged: the annotation says nothing about them.
          
 
          For MTBC this set was curated by hand. With a custom reference AMAP derives the equivalent automatically, which is what makes organism-agnostic mode work rather than merely run.
          
-
          What you must turn off#
          
-
          • --skip_typing: pathotypr assigns MTBC lineages from tuberculosis-specific marker SNPs. Meaningless elsewhere, and it will fail. AMAP warns when --genus is not Mycobacterium and this is not set.
          • --skip_mycobrowser: Mycobrowser functional categories are an MTBC resource. Auto-enabled when --ref_annotation is set and --genus is not Mycobacterium, but setting it explicitly is clearer.
          
-
          Also set --genus and --species to match your organism, since Bakta uses them for naming and reference selection.
          
-
          What stays the same#
          
+
          What you must turn off
          #
          
+
          • --skip_typing: pathotypr assigns MTBC lineages from tuberculosis-specific marker SNPs. Meaningless elsewhere, and it will fail. AMAP warns when --genus is not Mycobacterium and this is not set.
          • --skip_mycobrowser: Mycobrowser functional categories are an MTBC resource. Auto-enabled when --ref_annotation is set and --genus is not Mycobacterium, but setting it explicitly is clearer.
          
+
          Also set --genus and --species to match your organism, since Bakta uses them for naming and reference selection.
          
+
          What stays the same
          #
          
 
          Everything else. Case classification, conflict resolution, missing-gene rescue, the stop codon check, eggNOG functional annotation, duplication analysis, the pangenome and the report all work identically. The report simply omits the lineage and Mycobrowser sections, and the pangenome calls each genome by its sample ID where a typed run would use its lineage name.
          
-
          What to expect from a real run#
          
-
          E. coli O157:H7 Sakai corrected against K-12 MG1655, both straight from RefSeq, in the tutorial's run:
          
-
          reference proteins extracted from the K-12 GenBank   4318
+
          What to expect from aย real run
          #
          
+
          E.ย coli O157:H7 Sakai corrected against K-12 MG1655, both straight from RefSeq, in the tutorial's run:
          
+
          Output
          reference proteins extracted from the K-12 GenBank   4318
 CDS Bakta called on that same reference genome       4296
 Bakta false positives derived from the difference     142
 
@@ -126,36 +146,36 @@ 
          What to expect from a real runWorked example#
          
-
          E. coli K-12 MG1655 against the RefSeq reference:
          
-
          # Fetch the reference
-datasets download genome accession GCF_000005845.2 --include genome,gff3
-unzip ncbi_dataset.zip
-
-nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv \
-    --skip_typing --skip_mycobrowser \
-    --genus Escherichia --species "Escherichia coli" \
-    --ref_annotation ncbi_dataset/data/GCF_000005845.2/genomic.gff \
-    --ref_genome ncbi_dataset/data/GCF_000005845.2/GCF_000005845.2_ASM584v2_genomic.fna
          
-
          Things that trip people up#
          
-
          The annotation and the genome must match
          Coordinates from the annotation are used directly against the FASTA. An annotation from a different assembly of the same organism produces nonsense: extracted proteins will not translate, and the reference proteome will be mostly empty. PREPARE_REFERENCE prints the count of extracted proteins, so check it looks like a genome's worth.
          
-
          Reference genes need identifiers
          A GFF3 whose CDS features carry no locus_tag, ID, gene or Name is skipped entirely, and the run fails with "No proteins extracted". This happens with minimal GFF3 exports from some annotation tools.
          
-
          Distance to the reference
          The correction expects your samples to be the same species as the reference. The further away they are, the more genes fall into case 1 (nothing in the reference to compare against) and the less AMAP has to say. It is a correction tool, not an ortholog finder.
          
-
          Next steps#
          
-
-
          Edit this page on GitHub
          
-
+
          The 142 are the automatically derived equivalent of the hand-curated MTBC false-positive list, and 1,387 caseย 1 genes is what you would expect between two E.ย coli strains that genuinely differ.
          
+
          If your caseย 2 comes out at or near zero, stop and look at the reference rather than the sample. Every gene missing an exact match by a constant offset points at the reference, not at 4,000 coincidences.
          
+
          Worked example
          #
          
+
          E.ย coli K-12 MG1655 against the RefSeq reference:
          
+
          Terminal
          # Fetch the reference
+$ datasets download genome accession GCF_000005845.2 --include genome,gff3
+$ unzip ncbi_dataset.zip
+
+$ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv \
+    --skip_typing --skip_mycobrowser \
+    --genus Escherichia --species "Escherichia coli" \
+    --ref_annotation ncbi_dataset/data/GCF_000005845.2/genomic.gff \
+    --ref_genome ncbi_dataset/data/GCF_000005845.2/GCF_000005845.2_ASM584v2_genomic.fna
          
+
          Things that trip people up
          #
          
+
          WarningThe annotation and the genome must match
          Coordinates from the annotation are used directly against the FASTA. An annotation from a different assembly of the same organism produces nonsense: extracted proteins will not translate, and the reference proteome will be mostly empty. PREPARE_REFERENCE writes the count of extracted proteins into its work directory, where grep -h 'Curated proteins' work/*/*/.command.out finds it, so check it looks like a genome's worth.
          
+
          WarningReference genes need identifiers
          A GFF3 whose CDS features carry no locus_tag, ID, gene or Name is skipped entirely, and the run fails with "No proteins extracted". This happens with minimal GFF3 exports from some annotation tools.
          
+
          NoteDistance to the reference
          The correction expects your samples to be the same species as the reference. The further away they are, the more genes fall into caseย 1 (nothing in the reference to compare against) and the less AMAP has to say. It is a correction tool, not an ortholog finder.
          
+
          Next steps
          #
          
+
+
          Back to the top
          
+
          Edit this page on GitHub
          
+
 
          
-
 
 
-
 
 
 
diff --git a/docs/user-guide/custom-reference.md b/docs/user-guide/custom-reference.md
index 0b45477..0c073d8 100644
--- a/docs/user-guide/custom-reference.md
+++ b/docs/user-guide/custom-reference.md
@@ -1,7 +1,7 @@
-# Using your own reference
+# Custom reference
 
-AMAP corrects an annotation by refereeing it against a trusted one. For MTBC
-that trusted annotation is H37Rv, bundled in the image. Point
+AMAP corrects an annotation by refereeing it against a trusted one. For the
+*Mycobacterium tuberculosis* complex (MTBC) that trusted annotation is H37Rv, bundled in the image. Point
 `--ref_annotation` and `--ref_genome` at your own and the same machinery works
 for any prokaryote.
 
@@ -37,23 +37,29 @@ things.
 
 **1. Extracts the reference proteome.** Every CDS becomes a protein, headed
 `GENE_ID_START-END contig=SEQUENCE`, `SEQUENCE` being the name `--ref_genome`
-gives the sequence the gene is on. Those embedded coordinates are not
-decoration: the paralog step reads them back to predict where each protein
-belongs in the sample, which tells its own locus from a paralog's, so the
-header format matters. A CDS across the origin of a circular sequence ends past
-the sequence's last base, the way GFF3 writes it: tagA on plasmid pO157
-(92721 bp) of *E. coli* O157:H7 Sakai, `join(92527..92721,1..2502)` in its
-GenBank record, is `pO157p01_92527-95223`. From GenBank the existing
-`/translation` qualifier is used when present, otherwise the sequence is
-extracted and translated. From GFF3 the lines of each CDS are joined first, as
-GenBank's `join()` is, and translated with the bacterial code, reading
-`transl_except` (a selenocysteine) where it is given; on E. coli K-12 the two
-formats give the same 4318 proteins, and on Sakai the same 5155. A GFF3
-protein under 8 amino acids is left out.
-
-The gene ID comes from `locus_tag`, then `gene`, then `protein_id` for GenBank;
-`locus_tag`, `ID`, `gene`, then `Name` for GFF3. A CDS with none of these is
-skipped.
+gives the sequence the gene is on.
+
+- **The header matters.** The paralog step reads its coordinates back to
+  predict where each protein belongs in the sample, which tells its own locus
+  from a paralog's.
+- **From GenBank**, the existing `/translation` qualifier is used when present;
+  otherwise the sequence is extracted and translated.
+- **From GFF3**, the lines of each CDS are joined first, as GenBank's `join()`
+  is, and translated with the bacterial code, reading `transl_except` (a
+  selenocysteine) where it is given. A GFF3 protein under 8 amino acids is left
+  out.
+- **The gene ID** comes from `locus_tag`, then `gene`, then `protein_id` for
+  GenBank; `locus_tag`, `ID`, `gene`, then `Name` for GFF3. A CDS with none of
+  these is skipped.
+
+> [!NOTE]
+> **Genes across the origin**
+>
+> A CDS across the origin of a circular sequence ends past the sequence's last
+> base, the way GFF3 writes it: tagA on plasmid pO157 (92,721 bp) of *E. coli*
+> O157:H7 Sakai, `join(92527..92721,1..2502)` in its GenBank record, is
+> `pO157p01_92527-95223`. On *E. coli* K-12 the two formats give the same 4,318
+> proteins, and on Sakai the same 5,155.
 
 **2. Annotates the reference genome with Bakta.** The same Bakta that will
 annotate your samples, run on a genome whose true annotation is known, keeping
@@ -99,7 +105,7 @@ sample ID where a typed run would use its lineage name.
 *E. coli* O157:H7 Sakai corrected against K-12 MG1655, both straight from
 RefSeq, in [the tutorial's run](../getting-started/tutorial.md#9-the-same-thing-for-another-organism):
 
-```text
+```text output
 reference proteins extracted from the K-12 GenBank   4318
 CDS Bakta called on that same reference genome       4296
 Bakta false positives derived from the difference     142
@@ -113,12 +119,12 @@ final annotation                                     5335 genes
 ```
 
 The 142 are the automatically derived equivalent of the hand-curated MTBC
-false-positive list, and 1387 case 1 genes is what you would expect between two
+false-positive list, and 1,387 case 1 genes is what you would expect between two
 *E. coli* strains that genuinely differ.
 
 If your case 2 comes out at or near zero, stop and look at the reference rather
 than the sample. Every gene missing an exact match by a constant offset points
-at the reference, not at 4000 coincidences.
+at the reference, not at 4,000 coincidences.
 
 ## Worked example
 
@@ -145,8 +151,10 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
 > Coordinates from the annotation are used directly against the FASTA. An
 > annotation from a different assembly of the same organism produces
 > nonsense: extracted proteins will not translate, and the reference proteome
-> will be mostly empty. `PREPARE_REFERENCE` prints the count of extracted
-> proteins, so check it looks like a genome's worth.
+> will be mostly empty. `PREPARE_REFERENCE` writes the count of extracted
+> proteins into its work directory, where
+> `grep -h 'Curated proteins' work/*/*/.command.out` finds it, so check it
+> looks like a genome's worth.
 
 > [!WARNING]
 > **Reference genes need identifiers**
@@ -165,5 +173,5 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
 
 ## Next steps
 
-- [How it works](../how-it-works/index.md): what the correction does with the reference
+- [Overview](../how-it-works/index.md): what the correction does with the reference
 - [Parameters](parameters.md): the full flag reference
diff --git a/docs/user-guide/outputs.html b/docs/user-guide/outputs.html
index 3604f47..b178bff 100644
--- a/docs/user-guide/outputs.html
+++ b/docs/user-guide/outputs.html
@@ -4,208 +4,233 @@
 
 
 Outputs | AMAP docs
-
+
 
+
 
 
 
-
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
          
   
          
     
-      AMAP
+      AMAP
     
     
   
          
 
          
 
          
-
+  
+  
+
          
 
          
-
          Docs / User guide
          
+
          Docs / User guide
          Describes AMAP on main, ahead of 1.0.1: what changed
          
 
          Outputs
          
-
          Everything lands under --outdir, results/ by default, in one subdirectory per pipeline stage. {sampleId} below is the sample's name as lineage typing leaves it, L4_G26689 for sample G26689, or the sample ID alone under --skip_typing.
          
-
          Where to look first#
          
-
          What you wantFile
          The whole analysis, interactiveresults/final_report.html
          Each genome and each gene, question by questionresults/amap_report.html
          Corrected annotation for a sampleresults/AMAP/{sampleId}/*-AMAP.gff3
          Proteins for downstream toolsresults/AMAP/{sampleId}/*-AMAP.clean.faa
          Everything known about every generesults/AMAP/{sampleId}/*-AMAPmastertable.tsv
          Tool versions for your methods sectionresults/pipeline_info/software_versions.yml
          
-
          The full tree#
          
-
          PATHOTYPR/
          PATHOTYPRLeft out with --skip_typing.
          BAKTA/
          BAKTA_ANNOTATORAlways written.
          CDS_CASES/
          TYPE_CASESAlways written.
          AMAP/
          FIX_CDS, then GET_OUTPUTS and GET_COUNTLeft out with --skip_functional_annotation only FIX_CDS's *.AMAP.tsv, .faa and .fna are left.
          DUPLICATIONS/
          GET_DUPLICATIONS, GET_ALLDUPLeft out with --skip_functional_annotation.
          PANGENE/
          PANGENELeft out with --skip_functional_annotation.
          final_report.html
          FINAL_REPORTLeft out with --skip_functional_annotation.
          pipeline_info/
          VALIDATE_INPUTS, SOFTWARE_VERSIONS, DUMP_PARAMS and Nextflow itselfAlways written.
          Each folder under results/, the processes of the metro map that write it, and the options that leave it out. The colour is the metro line of its process.
          
-
          results/
+
          Every folder and file, and what a healthy run looks like.
          
+
          5 min read
          
+
          On this page
          
+
+
          Everything lands under --outdir, results/ by default, in one subdirectory per pipeline stage. {lineage}_{sampleId} below is the sample's name as lineage typing leaves it, L4_G26689 for sample G26689, or the sample ID alone under --skip_typing.
          
+
          Where to look first
          #
          
+
          What you wantFile
          The whole analysis, after a run with eggNOG (the full report)results/final_report.html
          Each genome and each gene, question by question, in every run (the report by question)results/amap_report.html
          Corrected annotation for a sample, after a run with eggNOGresults/AMAP/{lineage}_{sampleId}/*-AMAP.gff3
          Proteins for downstream tools, after a run with eggNOGresults/AMAP/{lineage}_{sampleId}/*-AMAP.clean.faa
          Everything known about every gene, after a run with eggNOGresults/AMAP/{lineage}_{sampleId}/*-AMAPmastertable.tsv
          The corrected genes, their proteins and sequences, in every runresults/AMAP/{lineage}_{sampleId}/*.AMAP.tsv, .faa, .fna
          Tool versions for your methods sectionresults/pipeline_info/software_versions.yml
          
+
          The full tree
          #
          
+
          PATHOTYPR/
          PATHOTYPRLeft out with --skip_typing.
          BAKTA/
          BAKTA_ANNOTATORAlways written.
          CDS_CASES/
          TYPE_CASESAlways written.
          AMAP/
          FIX_CDS, then GET_OUTPUTS and GET_COUNTWith --skip_functional_annotation, only FIX_CDS's *.AMAP.tsv, .faa and .fna are written.
          DUPLICATIONS/
          GET_DUPLICATIONS, GET_ALLDUPLeft out with --skip_functional_annotation.
          PANGENE/
          PANGENELeft out with --skip_functional_annotation.
          final_report.html
          FINAL_REPORTLeft out with --skip_functional_annotation.
          amap_report.html
          AMAP_REPORTAlways written.
          pipeline_info/
          VALIDATE_INPUTS, SOFTWARE_VERSIONS, DUMP_PARAMS and Nextflow itselfAlways written.
          Each folder under results/, the processes of the metro map that write it, and the options that leave it out. The colour is the metro line of its process.
          
+
          results/
 โ”œโ”€โ”€ AMAP/
-โ”‚   โ””โ”€โ”€ {sampleId}/
-โ”‚       โ”œโ”€โ”€ *-AMAP.faa                  # Corrected proteins, all genes including pseudogenes
-โ”‚       โ”œโ”€โ”€ *-AMAP.clean.faa            # No pseudogenes, no internal stops
-โ”‚       โ”œโ”€โ”€ *-AMAP.fna                  # Corrected nucleotide sequences
-โ”‚       โ”œโ”€โ”€ *-AMAP.gff3                 # Corrected annotation, GFF3
-โ”‚       โ”œโ”€โ”€ *-AMAP.gbk                  # Corrected annotation, GenBank
-โ”‚       โ”œโ”€โ”€ *-AMAP.qc.tsv               # Genome size, GC%, coding density, annotation rate
-โ”‚       โ”œโ”€โ”€ *-AMAPmastertable.tsv       # Coordinates plus eggNOG plus Mycobrowser, one row per gene
-โ”‚       โ”œโ”€โ”€ *-AMAP.COG.tsv              # COG category counts
-โ”‚       โ””โ”€โ”€ *-AMAP.MycoBrowser.tsv      # Mycobrowser category counts
-โ”‚
+โ”‚   โ””โ”€โ”€ {lineage}_{sampleId}/
+โ”‚       โ”œโ”€โ”€ *.AMAP.tsv                  # Every run: the corrected genes, one row each
+โ”‚       โ”œโ”€โ”€ *.AMAP.faa                  # Every run: their proteins
+โ”‚       โ”œโ”€โ”€ *.AMAP.fna                  # Every run: their nucleotide sequences
+โ”‚       โ”œโ”€โ”€ *-AMAP.faa                  # Corrected proteins, pseudogenes included
+โ”‚       โ”œโ”€โ”€ *-AMAP.clean.faa            # No pseudogenes, no internal stops
+โ”‚       โ”œโ”€โ”€ *-AMAP.fna                  # Corrected nucleotide sequences
+โ”‚       โ”œโ”€โ”€ *-AMAP.tsv                  # The finished annotation, every feature
+โ”‚       โ”œโ”€โ”€ *-AMAP.gff3                 # Corrected annotation, GFF3
+โ”‚       โ”œโ”€โ”€ *-AMAP.gbk                  # Corrected annotation, GenBank
+โ”‚       โ”œโ”€โ”€ *-AMAP.qc.tsv               # Size, GC%, coding density, annotation rate
+โ”‚       โ”œโ”€โ”€ *-AMAPmastertable.tsv       # Every gene, with eggNOG and Mycobrowser
+โ”‚       โ”œโ”€โ”€ *-AMAP.COG.tsv              # COG category counts
+โ”‚       โ”œโ”€โ”€ *-AMAP.MycoBrowser.tsv      # Mycobrowser category counts
+โ”‚       โ””โ”€โ”€ *-AMAP.noRef.tsv            # The genes that name no reference gene
+โ”‚
 โ”œโ”€โ”€ BAKTA/
-โ”‚   โ””โ”€โ”€ {sampleId}/                     # The uncorrected Bakta annotation, for comparison
-โ”‚       โ”œโ”€โ”€ *.faa  *.gbff  *.gff3  *.tsv
+โ”‚   โ””โ”€โ”€ {lineage}_{sampleId}/           # Bakta's own annotation, uncorrected
+โ”‚       โ”œโ”€โ”€ *.gff3  *.gbff  *.embl  *.tsv  *.json
+โ”‚       โ”œโ”€โ”€ *.faa  *.ffn  *.fna
 โ”‚       โ”œโ”€โ”€ *.hypotheticals.faa  *.hypotheticals.tsv
-โ”‚       โ”œโ”€โ”€ *.png                       # Genome map
-โ”‚       โ””โ”€โ”€ *.txt                       # Bakta summary statistics
-โ”‚
+โ”‚       โ”œโ”€โ”€ *.inference.tsv
+โ”‚       โ”œโ”€โ”€ *.png  *.svg                # Genome map
+โ”‚       โ””โ”€โ”€ *.txt  *.log                # Bakta summary statistics and log
+โ”‚
 โ”œโ”€โ”€ CDS_CASES/
 โ”‚   โ”œโ”€โ”€ summary/
-โ”‚   โ”‚   โ””โ”€โ”€ *-summary.tsv               # Case counts, all samples
-โ”‚   โ””โ”€โ”€ {sampleId}/
-โ”‚       โ”œโ”€โ”€ *-case[1-4].tsv             # One table per case
-โ”‚       โ”œโ”€โ”€ *-rvmissing.tsv             # Reference genes Bakta did not find
-โ”‚       โ”œโ”€โ”€ *.html                      # Interactive overlap plots
-โ”‚       โ””โ”€โ”€ *_corrected.tsv             # Final corrected coordinates
-โ”‚
+โ”‚   โ”‚   โ””โ”€โ”€ *-summary.tsv               # Case counts, all samples
+โ”‚   โ””โ”€โ”€ {lineage}_{sampleId}/
+โ”‚       โ”œโ”€โ”€ *-case[1-4].tsv             # One table per case
+โ”‚       โ”œโ”€โ”€ *-rvmissing.tsv             # Reference genes Bakta missed, put back
+โ”‚       โ”œโ”€โ”€ *.html                      # Interactive overlap plots
+โ”‚       โ””โ”€โ”€ *_corrected.tsv             # As classified, before the stop codon check
+โ”‚
 โ”œโ”€โ”€ DUPLICATIONS/
-โ”‚   โ”œโ”€โ”€ {sampleId}/
-โ”‚   โ”‚   โ”œโ”€โ”€ duplications*               # Raw Diamond output, within-genome
-โ”‚   โ”‚   โ”œโ”€โ”€ duplications_*_groups.tsv   # Duplicated genes grouped into families
-โ”‚   โ”‚   โ””โ”€โ”€ *.dup                       # Duplicated protein sequences
-โ”‚   โ”œโ”€โ”€ homologous_communities.tsv      # Communities across all samples, with network metrics
-โ”‚   โ””โ”€โ”€ homologous_communities.html     # Interactive network
-โ”‚
+โ”‚   โ”œโ”€โ”€ {lineage}_{sampleId}/
+โ”‚   โ”‚   โ”œโ”€โ”€ duplications_*              # Diamond hits within the genome, sorted
+โ”‚   โ”‚   โ”œโ”€โ”€ duplications_*_groups.tsv   # Duplicated genes grouped into families
+โ”‚   โ”‚   โ”œโ”€โ”€ duplications_*_network.png  # The families drawn
+โ”‚   โ”‚   โ””โ”€โ”€ *_interactive.html          # The same, interactive
+โ”‚   โ”œโ”€โ”€ homologous_communities.tsv      # Communities across samples, with metrics
+โ”‚   โ””โ”€โ”€ homologous_communities.html     # Interactive network
+โ”‚
 โ”œโ”€โ”€ PANGENE/
-โ”‚   โ”œโ”€โ”€ pangenome.gfa                   # Pangenome graph of the whole run
-โ”‚   โ”œโ”€โ”€ pangenome.bubbles.txt           # Structural variation in the graph
-โ”‚   โ”œโ”€โ”€ pangenome.Rtab                  # Gene presence/absence, a column per sample
-โ”‚   โ””โ”€โ”€ pangenome.stat                  # Graph statistics
-โ”‚
+โ”‚   โ”œโ”€โ”€ pangenome.gfa                   # Pangenome graph of the whole run
+โ”‚   โ”œโ”€โ”€ pangenome.bubbles.txt           # Structural variation in the graph
+โ”‚   โ”œโ”€โ”€ pangenome.Rtab                  # Gene presence/absence, a column per sample
+โ”‚   โ””โ”€โ”€ pangenome.stat                  # Graph statistics
+โ”‚
 โ”œโ”€โ”€ PATHOTYPR/
-โ”‚   โ”œโ”€โ”€ typing/*_summary.tsv            # Lineage calls, one row per contig with markers
-โ”‚   โ””โ”€โ”€ fastas/*.fasta                  # Each genome as {lineage}_{sampleId}.fasta
-โ”‚
+โ”‚   โ”œโ”€โ”€ typing/*_summary.tsv            # Lineage calls, one row per contig with markers
+โ”‚   โ””โ”€โ”€ fastas/*.fasta                  # Each genome as {lineage}_{sampleId}.fasta
+โ”‚
 โ”œโ”€โ”€ pipeline_info/
-โ”‚   โ”œโ”€โ”€ execution_report_*.html         # Nextflow execution report
-โ”‚   โ”œโ”€โ”€ execution_timeline_*.html       # Timeline
-โ”‚   โ”œโ”€โ”€ pipeline_dag_*.html             # Dependency graph
-โ”‚   โ”œโ”€โ”€ execution_trace_*.txt           # Per-process resource usage
-โ”‚   โ”œโ”€โ”€ software_versions.yml / .tsv    # Exact tool versions
-โ”‚   โ”œโ”€โ”€ params.yml                      # Parameters used for this run
-โ”‚   โ”œโ”€โ”€ validation.log                  # Pre-flight validation results
-โ”‚   โ””โ”€โ”€ run_summary.txt                 # Status, duration, sample counts
-โ”‚
-โ”œโ”€โ”€ final_report.html                   # The full interactive report
-โ””โ”€โ”€ amap_report.html                    # The report by question, also without eggNOG
          
-
          The key files#
          
-
          *-AMAP.gff3 and *-AMAP.gbk#
          
-
          The corrected annotation, which is the point of the pipeline. The GFF3 is spec-compliant: gene features parent their CDS features, attributes are URL-escaped, ##sequence-region pragmas are emitted per contig, and eggNOG annotation rides along as Ontology_term, eC_number, COG_category, KEGG_ko, KEGG_Pathway and PFAMs. The sequences are appended after ##FASTA.
          
+โ”‚   โ”œโ”€โ”€ execution_report_*.html         # Nextflow execution report
+โ”‚   โ”œโ”€โ”€ execution_timeline_*.html       # Timeline
+โ”‚   โ”œโ”€โ”€ pipeline_dag_*.html             # Dependency graph
+โ”‚   โ”œโ”€โ”€ execution_trace_*.txt           # Per-process resource usage
+โ”‚   โ”œโ”€โ”€ software_versions.yml / .tsv    # Exact tool versions
+โ”‚   โ”œโ”€โ”€ params.yml                      # Parameters used for this run
+โ”‚   โ”œโ”€โ”€ validation.log                  # The checks' verdict, when they pass
+โ”‚   โ””โ”€โ”€ run_summary.txt                 # Status, duration, task counts and the command
+โ”‚
+โ”œโ”€โ”€ final_report.html                   # The full report, after eggNOG
+โ””โ”€โ”€ amap_report.html                    # The report by question, in every run
          
+
          The .AMAP. files, with a dot, are the correction's own output, written in every run, --skip_functional_annotation included. The -AMAP. files, with a hyphen, are the finished annotation, written after eggNOG. *.AMAP.faa and *-AMAP.faa hold the same proteins, and the two .fna files the same sequences.
          
+
          The key files
          #
          
+
          *-AMAP.gff3 and *-AMAP.gbk
          #
          
+
          The corrected annotation, which is the point of the pipeline. The GFF3 is spec-compliant: gene features parent their CDS features, attributes are URL-escaped, ##sequence-region pragmas are emitted per contig, and eggNOG annotation rides along as Ontology_term, eC_number, COG_category, KEGG_ko, KEGG_Pathway and PFAMs. The sequences are appended after ##FASTA.
          
 
          Two kinds of gene carry marks, written the way NCBI writes them:
          
-
          • Pseudogenes have pseudo=true in the GFF3 and /pseudo in the GenBank, with the reason in Note and /note: internal stop codons, a frameshift or lost stop that makes the frame run on past the end of the reference protein, no stop at all, or a stop the reference itself lacks. A /pseudo CDS has no /translation.
          • Genes a contig end cut short, in a draft assembly, have partial=true, with start_range=.,START or end_range=END,. on the open end, and < or > on that end of the GenBank location. They are not pseudogenes. Each piece of a cut gene is there under the gene's name, the first in the assembly with the gene's identifier and the others with _2, _3.
          
-
          The same three facts, pseudogene, partial (5', 3' or both) and note, are the last columns of the master table and of *-AMAP.tsv.
          
+
          • Pseudogenes have pseudo=true in the GFF3 and /pseudo in the GenBank, with the reason in Note and /note: internal stop codons, a frameshift or lost stop that makes the frame run on past the end of the reference protein, no stop at all, or a stop the reference itself lacks. A /pseudo CDS has no /translation.
          • Genes a contig end cut short, in a draft assembly, have partial=true, with start_range=.,START or end_range=END,. on the open end, and < or > on that end of the GenBank location. They are not pseudogenes. Each piece of a cut gene is there under the gene's name, the first in the assembly with the gene's identifier and the others with _2, _3.
          
+
          The same three facts, pseudogene, partial (5', 3' or both) and note, are the last columns of the master table and of *-AMAP.tsv.
          
 
          Real lines: CDC1551 cut into 40 contigs.
          contig_4   AMAP  CDS  164980  165406  .  -  0  ID=L4_CDC1551d_Rv0354c;…;Dbxref=AMAP_ids:Rv0354c;pseudo=true;Note=frameshift or lost stop codon: its frame runs on 41 residues past the end of the reference protein
 contig_9   AMAP  CDS  4160    4591    .  -  0  ID=L4_CDC1551d_Rv1239c;…;Dbxref=AMAP_ids:Rv1239c;partial=true;end_range=4591,.
-contig_10  AMAP  CDS  101478  102146  .  +  0  ID=L4_CDC1551d_Rv1239c_2;…;Dbxref=AMAP_ids:Rv1239c;partial=true;end_range=102146,.
          pseudo=true a pseudogene, the reason in Notepartial=true cut by its contig's end, the open end in end_range or start_range
          … stands for the attributes left out; the marks are written the way NCBI writes them.
          
-
          A gene across the origin of a circular sequence is written the way Bakta writes it. Bakta takes a sequence for a circular replicon when its FASTA description says complete, as NCBI's "complete genome" and "complete sequence" do, or circular=true, as Unicycler's does. In *-AMAP.tsv and the master table such a gene has its start after its end (stcE on plasmid pO157 of E. coli O157:H7 Sakai is 92563 to 2502); in the GenBank it is join(92563..92721,1..2502) in a record marked circular; and in the GFF3 it ends past the sequence's last base, by the sequence's length (92563 to 95223 on the 92721 bp plasmid), on a region feature marked Is_circular=true, as the GFF3 specification and Bakta's own GFF3 have it.
          
-
          *-AMAP.faa versus *-AMAP.clean.faa#
          
-
          *-AMAP.faa mirrors the annotation exactly, pseudogenes included, so gene counts match the GFF3.
          
-
          *-AMAP.clean.faa drops the genes AMAP calls pseudogenes, those with pseudo=true in the GFF3, and any sequence with an internal stop codon, and strips the terminal stop. This is the file for orthology, phylogenomics, pangenome tools and dN/dS, all of which expect valid translations. It has fewer sequences than the GFF3 has genes, by design.
          
-
          *-AMAPmastertable.tsv#
          
-
          One row per gene, joining the corrected coordinates, the Bakta annotation, the eggNOG assignment (COG category, description, preferred name, GO, EC, KEGG, PFAMs, CAZy) and, for MTBC, the Mycobrowser functional category. This is the table to load when you want to ask a question about the annotation.
          
-
          *-AMAP.qc.tsv#
          
-
          One row per sample: genome size, contig count, N50, GC%, total CDS, how many of them AMAP calls pseudogenes (those with pseudo=true in the GFF3), hypothetical protein percentage, mean and median gene length, coding density, and the eggNOG annotation rate. Useful for spotting the sample that went wrong.
          
-
          CDS_CASES/#
          
-
          The audit trail for the correction. Each case table lists the genes that fell into it, so you can see exactly what was changed and why. The correction algorithm explains the cases; *-summary.tsv aggregates the counts across samples and feeds the report. Its last three columns say which sample the counts are for: Sequence is the name its files carry (L4_G26689), Lineage what typing called it (Untyped under --skip_typing) and Sample its ID in the sample sheet (G26689).
          
-
          What a healthy summary looks like#
          
-
          Two real runs, those of the tutorial, to calibrate expectations. On the left, M. tuberculosis CDC1551 against the bundled H37Rv reference. On the right, E. coli O157:H7 Sakai against K-12 MG1655, using a custom reference.
          
-
          CaseCDC1551 vs H37RvSakai vs K-12
          case2, exact match28133324
          case3, length differs2431
          case4, resolved conflict1100507
          case1_kept, genuinely novel111387
          rv_missing, rescued97133
          total_cds before correction40775367
          
-
          CDC1551 vs H37Rv4,077 Bakta calls
          case 2 2,813case 3 24case 4 1,100case 1, kept 11missing, put back 97
          Sakai vs K-125,367 Bakta calls
          case 2 3,324case 3 31case 4 507case 1, kept 1,387missing, put back 133
          The same table, drawn. The number that differs by two orders of magnitude is case 1: the prophages and the LEE island that O157:H7 carries and K-12 does not.
          
-
          The one number that differs by two orders of magnitude is case 1, and that is biology: CDC1551 is the same species as the reference so almost nothing is genuinely new, while O157:H7 carries prophages and the LEE island that K-12 does not have.
          
-
          What should worry you is the opposite: case 2 at or near zero. Every gene disagreeing with the reference by a few bases means something systematic is wrong with the reference, not with 4000 individual genes. The usual cause is a reference proteome whose coordinates do not describe the sequence its proteins were translated from.
          
-
          PANGENE/pangenome.Rtab#
          
-
          The gene presence/absence matrix of the whole run, the one both reports draw the pangenome from: one row per gene, one column per sample, 1 where pangene found the gene in that sample's genome. Every genome is searched with the same proteins, the reference proteome and the genes no reference gene matches, so a gene with a 1 in every column is one all the samples share. A gene is named after its reference gene (Rv1506c) or, when no reference gene matches it, after one sample's copy of it (L4_CDC1551_01396, the ID in that sample's master table).
          
-
          final_report.html#
          
+                     /locus_tag="L4_CDC1551d_Rv1239c_2"
          /pseudo no /translation, the reason in /note>102146 the 3' end runs off the contig (< marks a cut 5' end)
          โ€ฆ stands for the attributes left out; the marks are written the way NCBI writes them.
          
+
          A gene across the origin of a circular sequence is written the way Bakta writes it. Bakta takes a sequence for a circular replicon when its FASTA description says complete, as NCBI's "complete genome" and "complete sequence" do, or circular=true, as Unicycler's does. In *-AMAP.tsv and the master table such a gene has its start after its end (stcE on plasmid pO157 of E.ย coli O157:H7 Sakai is 92,563ย toย 2,502); in the GenBank it is join(92563..92721,1..2502) in a record marked circular; and in the GFF3 it ends past the sequence's last base, by the sequence's length (92,563ย toย 95,223 on the 92,721ย bp plasmid), on a region feature marked Is_circular=true, as the GFF3 specification and Bakta's own GFF3 have it.
          
+
          *-AMAP.faa versus *-AMAP.clean.faa
          #
          
+
          *-AMAP.faa mirrors the annotation exactly, pseudogenes included, so gene counts match the GFF3.
          
+
          *-AMAP.clean.faa drops the genes AMAP calls pseudogenes, those with pseudo=true in the GFF3, and any sequence with an internal stop codon, and strips the terminal stop. This is the file for orthology, phylogenomics, pangenome tools and dN/dS, all of which expect valid translations. It has fewer sequences than the GFF3 has genes, by design.
          
+
          *-AMAPmastertable.tsv
          #
          
+
          One row per gene, joining the corrected coordinates, the Bakta annotation, the eggNOG assignment (COG category, description, preferred name, GO, EC, KEGG, PFAMs, CAZy) and, for the Mycobacterium tuberculosis complex (MTBC), the Mycobrowser functional category. This is the table to load when you want to ask a question about the annotation.
          
+
          *-AMAP.qc.tsv
          #
          
+
          One row per sample: genome size, contig count, N50, GC%, total CDS, how many of them AMAP calls pseudogenes (those with pseudo=true in the GFF3), hypothetical protein percentage, mean and median gene length, coding density, and the eggNOG annotation rate. Useful for spotting the sample that went wrong.
          
+
          CDS_CASES/
          #
          
+
          The audit trail for the correction. Each case table lists the genes that fell into it, so you can see exactly what was changed and why. The correction algorithm explains the cases; *-summary.tsv aggregates the counts across samples and feeds the report. Its last three columns say which sample the counts are for: Sequence is the name its files carry (L4_G26689), Lineage what typing called it (Untyped under --skip_typing) and Sample its ID in the sample sheet (G26689).
          
+
          What aย healthy summary looks like
          #
          
+
          Two real runs, those of the tutorial, to calibrate expectations. On the left, M.ย tuberculosis CDC1551 against the bundled H37Rv reference. On the right, E.ย coli O157:H7 Sakai against K-12 MG1655, using a custom reference.
          
+
          CaseCDC1551 against H37RvSakai against K-12
          case2, perfect match2,8133,324
          case3, imperfect match2431
          case4, conflicting overlap, resolved1,100507
          case1_kept, genuinely novel111,387
          rv_missing, rescued97133
          total_cds before correction4,0775,367
          
+
          CDC1551 against H37Rv4,077 Bakta calls
          caseย 2 2,813caseย 3 24caseย 4 1,100caseย 1, kept 11missing, put back 97
          Sakai against K-125,367 Bakta calls
          caseย 2 3,324caseย 3 31caseย 4 507caseย 1, kept 1,387missing, put back 133
          The same table, drawn. The number that differs by two orders of magnitude is caseย 1: the prophages and the LEE island that O157:H7 carries and K-12 does not.
          
+
          The one number that differs by two orders of magnitude is caseย 1, and that is biology: CDC1551 is the same species as the reference so almost nothing is genuinely new, while O157:H7 carries prophages and the LEE island that K-12 does not have.
          
+
          What should worry you is the opposite: caseย 2 at or near zero. Every gene disagreeing with the reference by a few bases means something systematic is wrong with the reference, not with 4,000 individual genes. The usual cause is a reference proteome whose coordinates do not describe the sequence its proteins were translated from.
          
+
          PANGENE/pangenome.Rtab
          #
          
+
          The gene presence/absence matrix of the whole run, the one both reports draw the pangenome from: one row per gene, one column per sample, 1 where pangene found the gene in that sample's genome. Every genome is searched with the same proteins, the reference proteome and the genes no reference gene matches, so a gene with a 1 in every column is one all the samples share. A gene is named after its reference gene (Rv1506c) or, when no reference gene matches it, after one sample's copy of it (L4_CDC1551_01396, the ID in that sample's master table).
          
+
          final_report.html
          #
          
 
          Self-contained, so it travels by email. Covers per-sample quality, missing reference genes by lineage, PE/PPE family distribution, KEGG pathways, duplication communities and network topology, the pangenome (core, softcore, shell, cloud, frequency spectrum, presence/absence heatmaps), COG and Mycobrowser distributions, case statistics and the associated tests, plus a searchable master-table explorer.
          
-
          amap_report.html#
          
-
          One self-contained file, organised by the questions a run raises: how each genome annotated, where genes are missing or broken on the reference, one gene in every genome, what the correction changed, the pseudogenes by lineage, what lies outside the reference, and the functional profile. Written with or without the functional annotation. The interactive report goes through it.
          
-
          Effect of the skip options#
          
-
          • --skip_typing: No PATHOTYPR/ directory. Samples keep their original IDs instead of being renamed by lineage.
          • --skip_functional_annotation: The run stops after correction. No DUPLICATIONS/, PANGENE/ or final_report.html, and AMAP/ holds only the corrected table and sequences, *.AMAP.tsv, *.AMAP.faa and *.AMAP.fna, without the GFF3, the GenBank or the master table. You still get BAKTA/ and CDS_CASES/, including *_corrected.tsv with the corrected coordinates, and amap_report.html, without its eggNOG sections.
          • --skip_mycobrowser: *-AMAP.MycoBrowser.tsv is written empty and the Mycobrowser sections drop out of the report. Everything else is unaffected.
          
-
          Next steps#
          
-
-
          Edit this page on GitHub
          
-
+
          amap_report.html
          #
          
+
          One self-contained file, organised by the questions a run raises: how each genome annotated, where genes are missing or broken on the reference, one gene in every genome, what the correction changed, the pseudogenes by lineage, what lies outside the reference, and the functional profile. Written with or without the functional annotation. The report by question goes through it.
          
+
          Effect of the skip options
          #
          
+
          • --skip_typing: No PATHOTYPR/ directory. Samples keep their original IDs instead of being renamed by lineage.
          • --skip_functional_annotation: The run stops after correction. No DUPLICATIONS/, PANGENE/ or final_report.html, and AMAP/ holds only the corrected table and sequences, *.AMAP.tsv, *.AMAP.faa and *.AMAP.fna, without the GFF3, the GenBank or the master table. You still get BAKTA/ and CDS_CASES/, including *_corrected.tsv with the classification's coordinates, and amap_report.html, without its eggNOG sections.
          • --skip_mycobrowser: *-AMAP.MycoBrowser.tsv is written empty and the Mycobrowser sections drop out of the report. Everything else is unaffected.
          
+
          Next steps
          #
          
+
+
          Back to the top
          
+
          Edit this page on GitHub
          
+
 
          
-
 
 
-
 
-
-
 
 
diff --git a/docs/user-guide/outputs.md b/docs/user-guide/outputs.md
index acd1300..84ecc75 100644
--- a/docs/user-guide/outputs.md
+++ b/docs/user-guide/outputs.md
@@ -1,19 +1,20 @@
 # Outputs
 
 Everything lands under `--outdir`, `results/` by default, in one subdirectory
-per pipeline stage. `{sampleId}` below is the sample's name as lineage typing
-leaves it, `L4_G26689` for sample `G26689`, or the sample ID alone under
-`--skip_typing`.
+per pipeline stage. `{lineage}_{sampleId}` below is the sample's name as
+lineage typing leaves it, `L4_G26689` for sample `G26689`, or the sample ID
+alone under `--skip_typing`.
 
 ## Where to look first
 
 | What you want | File |
 | :--- | :--- |
-| The whole analysis, interactive | `results/final_report.html` |
-| Each genome and each gene, question by question | `results/amap_report.html` |
-| Corrected annotation for a sample | `results/AMAP/{sampleId}/*-AMAP.gff3` |
-| Proteins for downstream tools | `results/AMAP/{sampleId}/*-AMAP.clean.faa` |
-| Everything known about every gene | `results/AMAP/{sampleId}/*-AMAPmastertable.tsv` |
+| The whole analysis, after a run with eggNOG ([the full report](report.md)) | `results/final_report.html` |
+| Each genome and each gene, question by question, in every run ([the report by question](amap-report.md)) | `results/amap_report.html` |
+| Corrected annotation for a sample, after a run with eggNOG | `results/AMAP/{lineage}_{sampleId}/*-AMAP.gff3` |
+| Proteins for downstream tools, after a run with eggNOG | `results/AMAP/{lineage}_{sampleId}/*-AMAP.clean.faa` |
+| Everything known about every gene, after a run with eggNOG | `results/AMAP/{lineage}_{sampleId}/*-AMAPmastertable.tsv` |
+| The corrected genes, their proteins and sequences, in every run | `results/AMAP/{lineage}_{sampleId}/*.AMAP.tsv`, `.faa`, `.fna` |
 | Tool versions for your methods section | `results/pipeline_info/software_versions.yml` |
 
 ## The full tree
@@ -23,39 +24,47 @@ leaves it, `L4_G26689` for sample `G26689`, or the sample ID alone under
 ```text
 results/
 โ”œโ”€โ”€ AMAP/
-โ”‚   โ””โ”€โ”€ {sampleId}/
-โ”‚       โ”œโ”€โ”€ *-AMAP.faa                  # Corrected proteins, all genes including pseudogenes
+โ”‚   โ””โ”€โ”€ {lineage}_{sampleId}/
+โ”‚       โ”œโ”€โ”€ *.AMAP.tsv                  # Every run: the corrected genes, one row each
+โ”‚       โ”œโ”€โ”€ *.AMAP.faa                  # Every run: their proteins
+โ”‚       โ”œโ”€โ”€ *.AMAP.fna                  # Every run: their nucleotide sequences
+โ”‚       โ”œโ”€โ”€ *-AMAP.faa                  # Corrected proteins, pseudogenes included
 โ”‚       โ”œโ”€โ”€ *-AMAP.clean.faa            # No pseudogenes, no internal stops
 โ”‚       โ”œโ”€โ”€ *-AMAP.fna                  # Corrected nucleotide sequences
+โ”‚       โ”œโ”€โ”€ *-AMAP.tsv                  # The finished annotation, every feature
 โ”‚       โ”œโ”€โ”€ *-AMAP.gff3                 # Corrected annotation, GFF3
 โ”‚       โ”œโ”€โ”€ *-AMAP.gbk                  # Corrected annotation, GenBank
-โ”‚       โ”œโ”€โ”€ *-AMAP.qc.tsv               # Genome size, GC%, coding density, annotation rate
-โ”‚       โ”œโ”€โ”€ *-AMAPmastertable.tsv       # Coordinates plus eggNOG plus Mycobrowser, one row per gene
+โ”‚       โ”œโ”€โ”€ *-AMAP.qc.tsv               # Size, GC%, coding density, annotation rate
+โ”‚       โ”œโ”€โ”€ *-AMAPmastertable.tsv       # Every gene, with eggNOG and Mycobrowser
 โ”‚       โ”œโ”€โ”€ *-AMAP.COG.tsv              # COG category counts
-โ”‚       โ””โ”€โ”€ *-AMAP.MycoBrowser.tsv      # Mycobrowser category counts
+โ”‚       โ”œโ”€โ”€ *-AMAP.MycoBrowser.tsv      # Mycobrowser category counts
+โ”‚       โ””โ”€โ”€ *-AMAP.noRef.tsv            # The genes that name no reference gene
 โ”‚
 โ”œโ”€โ”€ BAKTA/
-โ”‚   โ””โ”€โ”€ {sampleId}/                     # The uncorrected Bakta annotation, for comparison
-โ”‚       โ”œโ”€โ”€ *.faa  *.gbff  *.gff3  *.tsv
+โ”‚   โ””โ”€โ”€ {lineage}_{sampleId}/           # Bakta's own annotation, uncorrected
+โ”‚       โ”œโ”€โ”€ *.gff3  *.gbff  *.embl  *.tsv  *.json
+โ”‚       โ”œโ”€โ”€ *.faa  *.ffn  *.fna
 โ”‚       โ”œโ”€โ”€ *.hypotheticals.faa  *.hypotheticals.tsv
-โ”‚       โ”œโ”€โ”€ *.png                       # Genome map
-โ”‚       โ””โ”€โ”€ *.txt                       # Bakta summary statistics
+โ”‚       โ”œโ”€โ”€ *.inference.tsv
+โ”‚       โ”œโ”€โ”€ *.png  *.svg                # Genome map
+โ”‚       โ””โ”€โ”€ *.txt  *.log                # Bakta summary statistics and log
 โ”‚
 โ”œโ”€โ”€ CDS_CASES/
 โ”‚   โ”œโ”€โ”€ summary/
 โ”‚   โ”‚   โ””โ”€โ”€ *-summary.tsv               # Case counts, all samples
-โ”‚   โ””โ”€โ”€ {sampleId}/
+โ”‚   โ””โ”€โ”€ {lineage}_{sampleId}/
 โ”‚       โ”œโ”€โ”€ *-case[1-4].tsv             # One table per case
-โ”‚       โ”œโ”€โ”€ *-rvmissing.tsv             # Reference genes Bakta did not find
+โ”‚       โ”œโ”€โ”€ *-rvmissing.tsv             # Reference genes Bakta missed, put back
 โ”‚       โ”œโ”€โ”€ *.html                      # Interactive overlap plots
-โ”‚       โ””โ”€โ”€ *_corrected.tsv             # Final corrected coordinates
+โ”‚       โ””โ”€โ”€ *_corrected.tsv             # As classified, before the stop codon check
 โ”‚
 โ”œโ”€โ”€ DUPLICATIONS/
-โ”‚   โ”œโ”€โ”€ {sampleId}/
-โ”‚   โ”‚   โ”œโ”€โ”€ duplications*               # Raw Diamond output, within-genome
+โ”‚   โ”œโ”€โ”€ {lineage}_{sampleId}/
+โ”‚   โ”‚   โ”œโ”€โ”€ duplications_*              # Diamond hits within the genome, sorted
 โ”‚   โ”‚   โ”œโ”€โ”€ duplications_*_groups.tsv   # Duplicated genes grouped into families
-โ”‚   โ”‚   โ””โ”€โ”€ *.dup                       # Duplicated protein sequences
-โ”‚   โ”œโ”€โ”€ homologous_communities.tsv      # Communities across all samples, with network metrics
+โ”‚   โ”‚   โ”œโ”€โ”€ duplications_*_network.png  # The families drawn
+โ”‚   โ”‚   โ””โ”€โ”€ *_interactive.html          # The same, interactive
+โ”‚   โ”œโ”€โ”€ homologous_communities.tsv      # Communities across samples, with metrics
 โ”‚   โ””โ”€โ”€ homologous_communities.html     # Interactive network
 โ”‚
 โ”œโ”€โ”€ PANGENE/
@@ -75,13 +84,19 @@ results/
 โ”‚   โ”œโ”€โ”€ execution_trace_*.txt           # Per-process resource usage
 โ”‚   โ”œโ”€โ”€ software_versions.yml / .tsv    # Exact tool versions
 โ”‚   โ”œโ”€โ”€ params.yml                      # Parameters used for this run
-โ”‚   โ”œโ”€โ”€ validation.log                  # Pre-flight validation results
-โ”‚   โ””โ”€โ”€ run_summary.txt                 # Status, duration, sample counts
+โ”‚   โ”œโ”€โ”€ validation.log                  # The checks' verdict, when they pass
+โ”‚   โ””โ”€โ”€ run_summary.txt                 # Status, duration, task counts and the command
 โ”‚
-โ”œโ”€โ”€ final_report.html                   # The full interactive report
-โ””โ”€โ”€ amap_report.html                    # The report by question, also without eggNOG
+โ”œโ”€โ”€ final_report.html                   # The full report, after eggNOG
+โ””โ”€โ”€ amap_report.html                    # The report by question, in every run
 ```
 
+The `.AMAP.` files, with a dot, are the correction's own output, written in
+every run, `--skip_functional_annotation` included. The `-AMAP.` files, with a
+hyphen, are the finished annotation, written after eggNOG. `*.AMAP.faa` and
+`*-AMAP.faa` hold the same proteins, and the two `.fna` files the same
+sequences.
+
 ## The key files
 
 ### `*-AMAP.gff3` and `*-AMAP.gbk`
@@ -115,10 +130,10 @@ A gene across the origin of a circular sequence is written the way Bakta writes
 it. Bakta takes a sequence for a circular replicon when its FASTA description
 says `complete`, as NCBI's "complete genome" and "complete sequence" do, or
 `circular=true`, as Unicycler's does. In `*-AMAP.tsv` and the master table such
-a gene has its start after its end (stcE on plasmid pO157 of E. coli O157:H7
-Sakai is 92563 to 2502); in the GenBank it is `join(92563..92721,1..2502)` in a
+a gene has its start after its end (stcE on plasmid pO157 of *E. coli* O157:H7
+Sakai is 92,563 to 2,502); in the GenBank it is `join(92563..92721,1..2502)` in a
 record marked circular; and in the GFF3 it ends past the sequence's last base,
-by the sequence's length (92563 to 95223 on the 92721 bp plasmid), on a
+by the sequence's length (92,563 to 95,223 on the 92,721 bp plasmid), on a
 `region` feature marked `Is_circular=true`, as the GFF3 specification and
 Bakta's own GFF3 have it.
 
@@ -137,7 +152,7 @@ sequences than the GFF3 has genes, by design.
 
 One row per gene, joining the corrected coordinates, the Bakta annotation, the
 eggNOG assignment (COG category, description, preferred name, GO, EC, KEGG,
-PFAMs, CAZy) and, for MTBC, the Mycobrowser functional category. This is the
+PFAMs, CAZy) and, for the *Mycobacterium tuberculosis* complex (MTBC), the Mycobrowser functional category. This is the
 table to load when you want to ask a question about the annotation.
 
 ### `*-AMAP.qc.tsv`
@@ -165,14 +180,14 @@ calibrate expectations. On the left, *M. tuberculosis* CDC1551 against the
 bundled H37Rv reference. On the right, *E. coli* O157:H7 Sakai against K-12
 MG1655, using a custom reference.
 
-| Case | CDC1551 vs H37Rv | Sakai vs K-12 |
+| Case | CDC1551 against H37Rv | Sakai against K-12 |
 | :--- | ---: | ---: |
-| case2, exact match | 2813 | 3324 |
-| case3, length differs | 24 | 31 |
-| case4, resolved conflict | 1100 | 507 |
-| case1_kept, genuinely novel | 11 | 1387 |
+| case2, perfect match | 2,813 | 3,324 |
+| case3, imperfect match | 24 | 31 |
+| case4, conflicting overlap, resolved | 1,100 | 507 |
+| case1_kept, genuinely novel | 11 | 1,387 |
 | rv_missing, rescued | 97 | 133 |
-| total_cds before correction | 4077 | 5367 |
+| total_cds before correction | 4,077 | 5,367 |
 
 
 
@@ -183,7 +198,7 @@ does not have.
 
 What should worry you is the opposite: **case 2 at or near zero**. Every gene
 disagreeing with the reference by a few bases means something systematic is
-wrong with the reference, not with 4000 individual genes. The usual cause is a
+wrong with the reference, not with 4,000 individual genes. The usual cause is a
 reference proteome whose coordinates do not describe the sequence its proteins
 were translated from.
 
@@ -213,7 +228,7 @@ One self-contained file, organised by the questions a run raises: how each
 genome annotated, where genes are missing or broken on the reference, one gene
 in every genome, what the correction changed, the pseudogenes by lineage, what
 lies outside the reference, and the functional profile. Written with or without
-the functional annotation. [The interactive report](amap-report.md) goes
+the functional annotation. [The report by question](amap-report.md) goes
 through it.
 
 ## Effect of the skip options
@@ -224,12 +239,12 @@ through it.
   `final_report.html`, and `AMAP/` holds only the corrected table and sequences,
   `*.AMAP.tsv`, `*.AMAP.faa` and `*.AMAP.fna`, without the GFF3, the GenBank or
   the master table. You still get `BAKTA/` and `CDS_CASES/`, including
-  `*_corrected.tsv` with the corrected coordinates, and `amap_report.html`,
+  `*_corrected.tsv` with the classification's coordinates, and `amap_report.html`,
   without its eggNOG sections.
 - **`--skip_mycobrowser`**: `*-AMAP.MycoBrowser.tsv` is written empty and the Mycobrowser sections drop
   out of the report. Everything else is unaffected.
 
 ## Next steps
 
-- [How it works](../how-it-works/index.md): what produced all this
+- [Overview](../how-it-works/index.md): what produced all this
 - [Duplication metrics](../how-it-works/duplications.md): reading the network numbers
diff --git a/docs/user-guide/parameters.html b/docs/user-guide/parameters.html
index 103d8d3..2a51ddc 100644
--- a/docs/user-guide/parameters.html
+++ b/docs/user-guide/parameters.html
@@ -4,133 +4,152 @@
 
 
 Parameters | AMAP docs
-
+
 
+
 
 
 
-
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
          
   
          
     
-      AMAP
+      AMAP
     
     
   
          
 
          
 
          
-
+  
+  
+
          
 
          
-
          Docs / User guide
          
+
          Docs / User guide
          Describes AMAP on main, ahead of 1.0.1: what changed
          
 
          Parameters
          
-
          Every AMAP parameter, what it defaults to, and when you would change it. Run nextflow run PathoGenOmics-Lab/AMAP --help for the same list at the terminal.
          
-
          Pipeline parameters take two dashes (--tsv). Nextflow's own options take one (-profile, -resume).
          
-
          Input and output#
          
-
          ParameterDescriptionTypeDefaultRequired
          --tsvSample sheet with sampleId and fasta columns, tab separated. Not needed with -profile test.filesamples.tsvYes
          --outdirDirectory for results.stringresultsNo
          
-
          Skip options#
          
-
          ParameterDescriptionTypeDefault
          --skip_typingSkip pathotypr lineage typing. Required for non-MTBC organisms.booleanfalse
          --skip_functional_annotationStop after annotation correction. Skips eggNOG, duplications, pangenome and the report.booleanfalse
          --skip_mycobrowserSkip Mycobrowser functional categories. Auto-enabled when --genus is not Mycobacterium and a custom reference is set.booleanfalse
          
+
          Every option, its default, and when to change it.
          
+
          2 min read
          
+
          On this page
          
+
+
          Every AMAP parameter, what it defaults to, and when you would change it. Run nextflow run PathoGenOmics-Lab/AMAP --help for the same list at the terminal.
          
+
          Pipeline parameters take two dashes (--tsv). Nextflow's own options take one (-profile, -resume).
          
+
          Input and output
          #
          
+
          ParameterDescriptionTypeDefaultRequired
          --tsvSample sheet with sampleId and fasta columns, tab separated.filesamples.tsvYes, except with -profile test
          --outdirDirectory for results.stringresultsNo
          
+
          Skip options
          #
          
+
          ParameterDescriptionTypeDefault
          --skip_typingSkip pathotypr lineage typing. Required outside the Mycobacterium tuberculosis complex (MTBC).booleanfalse
          --skip_functional_annotationStop after the correction: no eggNOG, duplications, pangenome or final_report.html; amap_report.html is still written, without its eggNOG sections.booleanfalse
          --skip_mycobrowserSkip Mycobrowser functional categories. Auto-enabled when --genus is not Mycobacterium and a custom reference is set.booleanfalse
          
 
          Their effect on the output tree is covered under outputs.
          
-
          Annotation#
          
-
          ParameterDescriptionTypeDefault
          --genusGenus passed to Bakta.string"Mycobacterium"
          --speciesSpecies passed to Bakta.string"Mycobacterium tuberculosis"
          
+
          Annotation
          #
          
+
          ParameterDescriptionTypeDefault
          --genusGenus passed to Bakta.string"Mycobacterium"
          --speciesSpecies passed to Bakta.string"Mycobacterium tuberculosis"
          
 
          Bakta uses these to pick naming conventions and reference sets, so setting them correctly improves the baseline annotation that AMAP then corrects.
          
-
          Resources#
          
-
          ParameterDescriptionTypeDefault
          --max_cpusCeiling on CPUs for any single process.integerthe machine's CPUs
          --max_memoryCeiling on memory for any single process.string16.GB
          --max_timeCeiling on wall-clock time for any single process.string72.h
          --scratchUse node-local scratch for I/O-heavy processes. On by default in the HPC profiles.booleanfalse
          
+
          Resources
          #
          
+
          ParameterDescriptionTypeDefault
          --max_cpusCeiling on CPUs for any single process.integerthe machine's CPUs
          --max_memoryCeiling on memory for any single process.string16.GB
          --max_timeCeiling on wall-clock time for any single process.string72.h
          --scratchUse node-local scratch for I/O-heavy processes. On by default in the executor profiles.booleanfalse
          
 
          These are ceilings, not requests. Each process asks for what it needs and is then clamped to these values, so lowering them makes AMAP fit a smaller machine rather than making it run faster.
          
-
          The defaults are a single machine's. The HPC profiles raise them to 48 CPUs and 128 GB, -profile highmem to 48 CPUs, 256 GB and 120 h, and -profile test sets 4 CPUs, 8 GB and 1 h, in whatever order -profile names them.
          
+
          The defaults are a single machine's. The executor profiles raise them to 48ย CPUs and 128ย GB, -profile highmem to 48ย CPUs, 256ย GB and 120ย h, and -profile test sets 4ย CPUs, 8ย GB and 1ย h, in whatever order -profile names them.
          
 
          Memory scales on retry: a process killed by the out-of-memory killer is resubmitted with more, up to the ceiling.
          
-
          Validation and information#
          
-
          ParameterDescriptionTypeDefault
          --validate_onlyRun input validation and stop.booleanfalse
          --strict_validationStop on validation warnings too, not only on errors.booleanfalse
          --helpPrint the help text and exit.booleanfalse
          --versionPrint pipeline and Nextflow versions and exit.booleanfalse
          
-
          Custom reference#
          
+
          Validation and information
          #
          
+
          ParameterDescriptionTypeDefault
          --validate_onlyRun input validation and stop.booleanfalse
          --strict_validationStop on validation warnings too, not only on errors.booleanfalse
          --helpPrint the help text and exit.booleanfalse
          --versionPrint pipeline and Nextflow versions and exit.booleanfalse
          
+
          Custom reference
          #
          
 
          Supply both to annotate any prokaryote against your own trusted annotation.
          
-
          ParameterDescriptionTypeDefault
          --ref_annotationTrusted reference annotation, GFF3 or GenBank.filenull
          --ref_genomeTrusted reference genome, FASTA.filenull
          
+
          ParameterDescriptionTypeDefault
          --ref_annotationTrusted reference annotation, GFF3 or GenBank.filenull
          --ref_genomeTrusted reference genome, FASTA.filenull
          
 
          They must be given together; setting only one is an error. See custom reference for what AMAP does with them.
          
-
          Reference and database arguments#
          
-
          You almost certainly do not need these
          Every file below ships inside the Docker and Singularity images, under /reference/. Running with -profile docker or -profile singularity picks them up automatically.
          
+
          Reference and database arguments
          #
          
+
          NoteYou almost certainly do not need these
          Every file below ships inside the Docker and Singularity images, under /reference/. Running with -profile docker or -profile singularity picks them up automatically.
          
 
          They exist for running without a container, for swapping in updated reference files, or for adapting the pipeline by hand.
          
-
          ParameterDescriptionDefault inside the container
          --bakta_dbBakta database directory, inside the container or on the host./data/bakta_db/db-light
          --eggnog_dbPre-formatted eggNOG database directory.eggnog_db
          --pathotyper_snpsLineage marker SNPs for pathotypr./reference/snps_lineages.tsv
          --pathotyper_refReference FASTA for pathotypr./reference/MTB_ancestor_reference.fasta
          --miniprot_ref_aaReference proteome aligned onto each genome./reference/aa_H37Rv.translated.fas
          --miniprot_ref_bad_baktaKnown problematic annotations, used to filter novel-gene candidates./reference/bad.bakta.translated.fas
          --miniprot_ref_h37rvReference genome FASTA./reference/new_refH37Rv.fasta
          --bakta_proteinsReference proteins GenBank file for Bakta./reference/sequence.gb
          --mycobrowser_refMycobrowser functional categories table./reference/Mycobacterium_tuberculosis_H37Rv_txt_v4.txt
          
+
          ParameterDescriptionDefault inside the container
          --bakta_dbBakta database directory, inside the container or on the host./data/bakta_db/db-light
          --eggnog_dbPre-formatted eggNOG database directory.eggnog_db
          --pathotyper_snpsLineage marker SNPs for pathotypr./reference/snps_lineages.tsv
          --pathotyper_refReference FASTA for pathotypr./reference/MTB_ancestor_reference.fasta
          --miniprot_ref_aaReference proteome aligned onto each genome./reference/aa_H37Rv.translated.fas
          --miniprot_ref_bad_baktaKnown problematic annotations, used to filter novel-gene candidates./reference/bad.bakta.translated.fas
          --miniprot_ref_h37rvReference genome FASTA./reference/new_refH37Rv.fasta
          --bakta_proteinsReference proteins GenBank file for Bakta./reference/sequence.gb
          --mycobrowser_refMycobrowser functional categories table./reference/Mycobacterium_tuberculosis_H37Rv_txt_v4.txt
          
 
          A database or a file of your own on the host, the full Bakta database or a newer Mycobrowser table for instance, is mounted into the tasks that use it, so it works with the container profiles too. A path that is not on the host is read inside the container, where the defaults are, and validation stops a run whose file is in neither place.
          
-
          The eggNOG database is the one exception: it is not bundled, because of its size. AMAP downloads and builds it on the first run and caches it at --eggnog_db, so point that at somewhere persistent and shared if you run on a cluster.
          
-
          Nextflow options worth knowing#
          
+
          The eggNOG database is the one exception: it is not bundled, because of its size. AMAP downloads and builds it on the first run and caches it at --eggnog_db, so point that at somewhere persistent and shared if you run on a cluster.
          
+
          Nextflow options worth knowing
          #
          
 
          These belong to Nextflow rather than AMAP, so they take a single dash.
          
-
          OptionWhat it does
          -profileSelects execution profiles, comma separated: docker, singularity, slurm, sge, pbs, test.
          -resumeReuse cached results from the previous run.
          -work-dirWhere intermediate files go. Defaults to ./work, and it gets large.
          -process.queueOverride the scheduler queue or partition.
          -process.clusterOptionsExtra flags passed to the scheduler, such as '--account=mygroup'.
          
+
          OptionWhat it does
          -profileSelects execution profiles, comma separated: docker, singularity, standard, slurm, sge, pbs, lsf, test, highmem.
          -resumeReuse cached results from the previous run.
          -rRun a given branch, tag or full commit hash of AMAP; see pinning the pipeline.
          -work-dirWhere intermediate files go. Defaults to ./work, and it gets large.
          -process.queueOverride the scheduler queue or partition.
          -process.clusterOptionsExtra flags passed to the scheduler, such as '--account=mygroup'.
          
 
          See running AMAP for how the profiles combine.
          
-
          Edit this page on GitHub
          
-
+
          Back to the top
          
+
          Edit this page on GitHub
          
+
 
          
-
 
 
-
 
 
 
diff --git a/docs/user-guide/parameters.md b/docs/user-guide/parameters.md
index 097e352..159947f 100644
--- a/docs/user-guide/parameters.md
+++ b/docs/user-guide/parameters.md
@@ -10,15 +10,15 @@ Pipeline parameters take two dashes (`--tsv`). Nextflow's own options take one
 
 | Parameter | Description | Type | Default | Required |
 | :--- | :--- | :--- | :--- | :--- |
-| `--tsv` | Sample sheet with `sampleId` and `fasta` columns, tab separated. Not needed with `-profile test`. | `file` | `samples.tsv` | **Yes** |
+| `--tsv` | Sample sheet with `sampleId` and `fasta` columns, tab separated. | `file` | `samples.tsv` | Yes, except with `-profile test` |
 | `--outdir` | Directory for results. | `string` | `results` | No |
 
 ## Skip options
 
 | Parameter | Description | Type | Default |
 | :--- | :--- | :--- | :--- |
-| `--skip_typing` | Skip pathotypr lineage typing. Required for non-MTBC organisms. | `boolean` | `false` |
-| `--skip_functional_annotation` | Stop after annotation correction. Skips eggNOG, duplications, pangenome and the report. | `boolean` | `false` |
+| `--skip_typing` | Skip pathotypr lineage typing. Required outside the *Mycobacterium tuberculosis* complex (MTBC). | `boolean` | `false` |
+| `--skip_functional_annotation` | Stop after the correction: no eggNOG, duplications, pangenome or `final_report.html`; `amap_report.html` is still written, without its eggNOG sections. | `boolean` | `false` |
 | `--skip_mycobrowser` | Skip Mycobrowser functional categories. Auto-enabled when `--genus` is not Mycobacterium and a custom reference is set. | `boolean` | `false` |
 
 Their effect on the output tree is covered under
@@ -41,13 +41,13 @@ correctly improves the baseline annotation that AMAP then corrects.
 | `--max_cpus` | Ceiling on CPUs for any single process. | `integer` | the machine's CPUs |
 | `--max_memory` | Ceiling on memory for any single process. | `string` | `16.GB` |
 | `--max_time` | Ceiling on wall-clock time for any single process. | `string` | `72.h` |
-| `--scratch` | Use node-local scratch for I/O-heavy processes. On by default in the HPC profiles. | `boolean` | `false` |
+| `--scratch` | Use node-local scratch for I/O-heavy processes. On by default in the executor profiles. | `boolean` | `false` |
 
 These are ceilings, not requests. Each process asks for what it needs and is
 then clamped to these values, so lowering them makes AMAP fit a smaller machine
 rather than making it run faster.
 
-The defaults are a single machine's. The HPC profiles raise them to 48 CPUs
+The defaults are a single machine's. The executor profiles raise them to 48 CPUs
 and 128 GB, `-profile highmem` to 48 CPUs, 256 GB and 120 h, and
 `-profile test` sets 4 CPUs, 8 GB and 1 h, in whatever order `-profile` names
 them.
@@ -117,8 +117,9 @@ These belong to Nextflow rather than AMAP, so they take a single dash.
 
 | Option | What it does |
 | :--- | :--- |
-| `-profile` | Selects execution profiles, comma separated: `docker`, `singularity`, `slurm`, `sge`, `pbs`, `test`. |
+| `-profile` | Selects execution profiles, comma separated: `docker`, `singularity`, `standard`, `slurm`, `sge`, `pbs`, `lsf`, `test`, `highmem`. |
 | `-resume` | Reuse cached results from the previous run. |
+| `-r` | Run a given branch, tag or full commit hash of AMAP; see [pinning the pipeline](running.md#pinning-the-pipeline). |
 | `-work-dir` | Where intermediate files go. Defaults to `./work`, and it gets large. |
 | `-process.queue` | Override the scheduler queue or partition. |
 | `-process.clusterOptions` | Extra flags passed to the scheduler, such as `'--account=mygroup'`. |
diff --git a/docs/user-guide/report.html b/docs/user-guide/report.html
index 889a029..19984ad 100644
--- a/docs/user-guide/report.html
+++ b/docs/user-guide/report.html
@@ -3,148 +3,162 @@
 
 
 
-The report | AMAP docs
-
+The full report | AMAP docs
+
 
+
 
 
-
-
+
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
          
   
          
     
-      AMAP
+      AMAP
     
     
   
          
 
          
 
          
-
+  
+  
+
          
 
          
-
          Docs / User guide
          
-
          The report
          
-
          Every run ends in one file, results/final_report.html. It is self-contained: all the figures, tables and JavaScript are embedded, so it opens with no network and survives being emailed to someone who has never heard of Nextflow.
          
-
          To see what the sections look like before running anything, open the example report: a rendered report for twelve MTBC genomes across four lineages.
          
-
          The example uses generated data
          The twelve samples in it are synthetic. The distributions are drawn around a real M. tuberculosis CDC1551 run, so the shapes are representative, but no conclusion should be drawn from the numbers themselves.
          
-
          bin/report/demo/make_demo.py builds that dataset if you want to render the report yourself without waiting for a pipeline run.
          
-
          What is in it#
          
-
          final_report.html, section by sectionsynthetic data
          The overview section of the example reportOverviewDataset composition, then sample quality and annotation quality: where an outlier sample shows itself.The annotation results section of the example reportAnnotation resultsGenes per lineage, the genes the reference does not have, the correction summary and the reference genes Bakta missed.The functional annotation section of the example reportFunctional annotationCOG and Mycobrowser categories, the comparison with the reference, PE/PPE families, KEGG pathways and pseudogenes.The duplications section of the example reportDuplicationsCommunity sizes, the communities shared across genomes and their network metrics.The pangenome section of the example reportPangenomeCore and accessory genes, the gene frequency spectrum and the presence and absence of every gene.The cds statistics section of the example reportCDS statisticsHow many genes fall in each case, lineage by lineage: box and violin plots with Dunn's tests, the share of each case, a heatmap and a table per case.The master table section of the example reportMaster tableEvery gene of every sample in one table to search, sort and export.
          The example report's seven sections, in their order. Each picture opens the report there; its twelve genomes are made up, so read the layout, not the numbers.
          
-
          Overview#
          
+
          Docs / User guide
          Describes AMAP on main, ahead of 1.0.1: what changed
          
+
          The full report: final_report.html
          
+
          Each section of the full report, and what it shows.
          
+
          3 min read
          
+
          On this page
          
+
+
          A run with the functional annotation ends in results/final_report.html, the whole analysis in one file, built after eggNOG, the duplications and the pangenome. Every run, with or without them, also writes the report by question, results/amap_report.html. So a run with --skip_functional_annotation has no final_report.html; amap_report.html is still written, without its eggNOG sections.
          
+
          It is self-contained: all the figures, tables and JavaScript are embedded, so it opens with no network and survives being emailed to someone who has never heard of Nextflow.
          
+
          To see what the sections look like before running anything, open the example report: a rendered report for twelve genomes of the Mycobacterium tuberculosis complex (MTBC) across four lineages.
          
+
          NoteThe example uses generated data
          The twelve samples in it are synthetic. The distributions are drawn around a real M.ย tuberculosis CDC1551 run, so the shapes are representative, but no conclusion should be drawn from the numbers themselves.
          
+
          bin/report/demo/make_demo.py builds that dataset if you want to render the report yourself without waiting for a pipeline run.
          
+
          What is in it
          #
          
+
          final_report.html, section by sectionsynthetic data
          The overview section of the example reportOverviewDataset composition, then sample quality and annotation quality: where an outlier sample shows itself.The annotation results section of the example reportAnnotation resultsGenes per lineage, the genes the reference does not have, the correction summary and the reference genes Bakta missed.The functional annotation section of the example reportFunctional annotationCOG and Mycobrowser categories, the comparison with the reference, PE/PPE families, KEGG pathways and pseudogenes.The duplications section of the example reportDuplicationsCommunity sizes, the communities shared across genomes and their network metrics.The pangenome section of the example reportPangenomeCore and accessory genes, the gene frequency spectrum and the presence and absence of every gene.The CDS statistics section of the example reportCDS statisticsHow many genes fall in each case, lineage by lineage: box and violin plots with Dunn's tests, the share of each case, a heatmap and a table per case.The master table explorer section of the example reportMaster table explorerEvery gene of every sample in one table to search, sort and export.
          The example report's seven sections, in their order. Each picture opens the report there; its twelve genomes are made up, so read the layout, not the numbers.
          
+
          Overview
          #
          
 
          Dataset composition, then a per-sample quality panel: CDS counts, correction rates and missing reference genes. The annotation quality tab carries coding density, hypothetical percentage and genome size, which is where an outlier sample declares itself before you have gone any further.
          
-
          Annotation results#
          
+
          Annotation results
          #
          
 
          CDS distribution per lineage, the genes with no counterpart in the reference, and the correction summary: how many genes fell into each case and what the adjudication did with them. Read this alongside the correction algorithm.
          
 
          Missing reference genes get both a heatmap, for spotting a gene absent from a whole lineage rather than one sample, and a searchable table.
          
-
          Functional annotation#
          
+
          Functional annotation
          #
          
 
          COG categories from eggNOG, Mycobrowser functional categories for MTBC, and a comparison against the reference distribution. Each has its own per-category breakdown in counts and percentages, with the statistical tests alongside.
          
-
          PE/PPE gene families get their own section, since they are both the hardest part of an MTBC genome to annotate and usually the most interesting. The pseudogenes close it: how many AMAP called in each sample, lineage by lineage, a sample with none at 0, the same genes the QC table counts and the GFF3 marks pseudo=true. The master table's note column says why each is a pseudogene.
          
-
          KEGG pathways#
          
-
          The most represented metabolic pathways, and how they differ between lineages.
          
-
          Duplications#
          
+
          PE/PPE gene families get their own section, since they are both the hardest part of an MTBC genome to annotate and usually the most interesting. The KEGG pathways follow: the most represented metabolic pathways, and how they differ between lineages.
          
+
          The pseudogenes close it: how many AMAP called in each sample, lineage by lineage, a sample with none at 0, the same genes the QC table counts and the GFF3 marks pseudo=true. The master table's note column says why each is a pseudogene.
          
+
          Duplications
          #
          
 
          Community sizes, cross-genome homologous pairs and the network topology metrics. Duplication metrics explains what each number means.
          
-
          Pangenome#
          
+
          Pangenome
          #
          
 
          Core, softcore, shell and cloud classification, the gene frequency spectrum, and presence/absence heatmaps.
          
-
          CDS statistics#
          
+
          CDS statistics
          #
          
 
          How many genes fall in each case, lineage by lineage: box and violin plots with Dunn's tests between lineages, the share of each case in every lineage, a heatmap of the means, and a table per case with the count, mean, median and spread of each lineage.
          
-
          Master table explorer#
          
+
          Master table explorer
          #
          
 
          Every gene of every sample in one searchable, sortable table, with the functional annotation attached. Useful for answering a specific question without leaving the report.
          
-
          What changes when steps are skipped#
          
-
          • --skip_typing: Everything still renders, but every sample is grouped under Untyped, so the per-lineage comparisons have a single group and the statistical tests drop out.
          • --skip_mycobrowser: The Mycobrowser and PE/PPE sections are omitted. COG, KEGG, pangenome and duplications are unaffected. This is automatic for non-MTBC organisms.
          • --skip_functional_annotation: No report at all. The run stops after correction.
          
-
          Rendering it yourself#
          
-
          The report is bin/report/report_AMAP.Rmd, rendered by the FINAL_REPORT process. To iterate on it without running the pipeline:
          
-
          python3 bin/report/demo/make_demo.py demo_inputs
-
-docker run --rm -v "$PWD:/w" -w /w paururo/amap:latest \
-  Rscript -e 'rmarkdown::render("bin/report/report_AMAP.Rmd",
-    output_file="/w/demo.html",
-    params=list(version="v1.0.1",
-                cog_file="/w/demo_inputs/input_cog.txt",
-                myco_file="/w/demo_inputs/input_myco.txt",
-                summary_file="/w/demo_inputs/input_summary.txt",
-                empty_file="/w/demo_inputs/input_noref.txt",
-                rvmissing_file="/w/demo_inputs/input_rvmissing.txt",
-                mastertable_file="/w/demo_inputs/input_mastertables.txt",
-                pangene_file="/w/demo_inputs/input_pangene.txt",
-                dup_file="/w/demo_inputs/homologous_communities.tsv",
-                qc_file="/w/demo_inputs/input_qc.txt"))'
          
-
          Keep it self-contained
          Nothing in the Rmd, header.html or styles.css may point at a remote URL. pandoc will fetch and inline a remote resource when the machine rendering has a network, which hides the problem, and leave it dangling when it does not. A report rendered on an isolated compute node then arrives with a broken image and a browser trying to reach the internet.
          
+
          What changes when steps are skipped
          #
          
+
          • --skip_typing: Everything still renders, but every sample is grouped under Untyped, so the per-lineage comparisons have a single group and the statistical tests drop out.
          • --skip_mycobrowser: The Mycobrowser and PE/PPE sections are omitted. COG, KEGG, pangenome and duplications are unaffected. This is automatic for non-MTBC organisms.
          • --skip_functional_annotation: The run stops after correction: no final_report.html; amap_report.html is still written, without its eggNOG sections.
          
+
          Rendering it yourself
          #
          
+
          The report is bin/report/report_AMAP.Rmd, rendered by the FINAL_REPORT process. To iterate on it without running the pipeline:
          
+
          Terminal
          $ python3 bin/report/demo/make_demo.py demo_inputs
+
+$ docker run --rm -v "$PWD:/w" -w /w paururo/amap:latest \
+  Rscript -e 'rmarkdown::render("bin/report/report_AMAP.Rmd",
+    output_file="/w/demo.html",
+    params=list(
+      version="v1.0.1",
+      cog_file="/w/demo_inputs/input_cog.txt",
+      myco_file="/w/demo_inputs/input_myco.txt",
+      summary_file="/w/demo_inputs/input_summary.txt",
+      empty_file="/w/demo_inputs/input_noref.txt",
+      rvmissing_file="/w/demo_inputs/input_rvmissing.txt",
+      mastertable_file="/w/demo_inputs/input_mastertables.txt",
+      pangene_file="/w/demo_inputs/input_pangene.txt",
+      dup_file="/w/demo_inputs/homologous_communities.tsv",
+      qc_file="/w/demo_inputs/input_qc.txt"))'
          
+
          WarningKeep it self-contained
          Nothing in the Rmd, header.html or styles.css may point at a remote URL. Pandoc fetches and inlines a remote resource when the machine rendering has a network, which hides the problem, and leaves it dangling when it does not. A report rendered on an isolated compute node then arrives with a broken image and a browser trying to reach the internet.
          
 
          Both logos live next to the Rmd for this reason, and the image zoom is plain DOM rather than jQuery from a CDN.
          
-
          Edit this page on GitHub
          
-
+
          Edit this page on GitHub
          
+
 
          
-
 
 
-
 
 
 
diff --git a/docs/user-guide/report.md b/docs/user-guide/report.md
index 0b43828..723a85f 100644
--- a/docs/user-guide/report.md
+++ b/docs/user-guide/report.md
@@ -1,12 +1,19 @@
-# The report
+# The full report: final_report.html
 
-Every run ends in one file, `results/final_report.html`. It is self-contained:
-all the figures, tables and JavaScript are embedded, so it opens with no
-network and survives being emailed to someone who has never heard of Nextflow.
+A run with the functional annotation ends in `results/final_report.html`, the
+whole analysis in one file, built after eggNOG, the duplications and the
+pangenome. Every run, with or without them, also writes
+[the report by question](amap-report.md), `results/amap_report.html`. So a run
+with `--skip_functional_annotation` has no `final_report.html`;
+`amap_report.html` is still written, without its eggNOG sections.
+
+It is self-contained: all the figures, tables and JavaScript are embedded, so
+it opens with no network and survives being emailed to someone who has never
+heard of Nextflow.
 
 To see what the sections look like before running anything, open the
 [example report](../examples/demo_report.html): a rendered report for twelve
-MTBC genomes across four lineages.
+genomes of the *Mycobacterium tuberculosis* complex (MTBC) across four lineages.
 
 > [!NOTE]
 > **The example uses generated data**
@@ -46,14 +53,14 @@ comparison against the reference distribution. Each has its own per-category
 breakdown in counts and percentages, with the statistical tests alongside.
 
 PE/PPE gene families get their own section, since they are both the hardest
-part of an MTBC genome to annotate and usually the most interesting. The
-pseudogenes close it: how many AMAP called in each sample, lineage by lineage,
-a sample with none at 0, the same genes the QC table counts and the GFF3 marks
-`pseudo=true`. The master table's `note` column says why each is a pseudogene.
-
-### KEGG pathways
+part of an MTBC genome to annotate and usually the most interesting. The KEGG
+pathways follow: the most represented metabolic pathways, and how they differ
+between lineages.
 
-The most represented metabolic pathways, and how they differ between lineages.
+The pseudogenes close it: how many AMAP called in each sample, lineage by
+lineage, a sample with none at 0, the same genes the QC table counts and the
+GFF3 marks `pseudo=true`. The master table's `note` column says why each is a
+pseudogene.
 
 ### Duplications
 
@@ -86,7 +93,8 @@ without leaving the report.
   drop out.
 - **`--skip_mycobrowser`**: The Mycobrowser and PE/PPE sections are omitted. COG, KEGG, pangenome and
   duplications are unaffected. This is automatic for non-MTBC organisms.
-- **`--skip_functional_annotation`**: No report at all. The run stops after correction.
+- **`--skip_functional_annotation`**: The run stops after correction: no `final_report.html`;
+  `amap_report.html` is still written, without its eggNOG sections.
 
 ## Rendering it yourself
 
@@ -99,24 +107,25 @@ python3 bin/report/demo/make_demo.py demo_inputs
 docker run --rm -v "$PWD:/w" -w /w paururo/amap:latest \
   Rscript -e 'rmarkdown::render("bin/report/report_AMAP.Rmd",
     output_file="/w/demo.html",
-    params=list(version="v1.0.1",
-                cog_file="/w/demo_inputs/input_cog.txt",
-                myco_file="/w/demo_inputs/input_myco.txt",
-                summary_file="/w/demo_inputs/input_summary.txt",
-                empty_file="/w/demo_inputs/input_noref.txt",
-                rvmissing_file="/w/demo_inputs/input_rvmissing.txt",
-                mastertable_file="/w/demo_inputs/input_mastertables.txt",
-                pangene_file="/w/demo_inputs/input_pangene.txt",
-                dup_file="/w/demo_inputs/homologous_communities.tsv",
-                qc_file="/w/demo_inputs/input_qc.txt"))'
+    params=list(
+      version="v1.0.1",
+      cog_file="/w/demo_inputs/input_cog.txt",
+      myco_file="/w/demo_inputs/input_myco.txt",
+      summary_file="/w/demo_inputs/input_summary.txt",
+      empty_file="/w/demo_inputs/input_noref.txt",
+      rvmissing_file="/w/demo_inputs/input_rvmissing.txt",
+      mastertable_file="/w/demo_inputs/input_mastertables.txt",
+      pangene_file="/w/demo_inputs/input_pangene.txt",
+      dup_file="/w/demo_inputs/homologous_communities.tsv",
+      qc_file="/w/demo_inputs/input_qc.txt"))'
 ```
 
 > [!WARNING]
 > **Keep it self-contained**
 >
 > Nothing in the Rmd, `header.html` or `styles.css` may point at a remote
-> URL. pandoc will fetch and inline a remote resource when the machine
-> rendering has a network, which hides the problem, and leave it dangling
+> URL. Pandoc fetches and inlines a remote resource when the machine
+> rendering has a network, which hides the problem, and leaves it dangling
 > when it does not. A report rendered on an isolated compute node then
 > arrives with a broken image and a browser trying to reach the internet.
 >
diff --git a/docs/user-guide/running.html b/docs/user-guide/running.html
index 5360303..3ea1ff2 100644
--- a/docs/user-guide/running.html
+++ b/docs/user-guide/running.html
@@ -4,149 +4,182 @@
 
 
 Running AMAP | AMAP docs
-
+
 
+
 
 
 
-
+
 
 
 
 
 
 
-
+
 
 
-
+
 
-
+
 
 
 
          
   
          
     
-      AMAP
+      AMAP
     
     
   
          
 
          
 
          
-
+  
+  
+
          
 
          
-
          Docs / User guide
          
+
          Docs / User guide
          Describes AMAP on main, ahead of 1.0.1: what changed
          
 
          Running AMAP
          
+
          Profiles, schedulers and larger runs.
          
+
          3 min read
          
+
          On this page
          
+
 
          Nextflow runs the pipeline straight from GitHub, so there is nothing to clone and nothing to keep up to date by hand.
          
-
          nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
          
-
          Profiles#
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
          
+
          Profiles
          #
          
 
          A profile is a bundle of settings. You normally combine one container profile with one executor profile.
          
-
          ProfileKindWhat it does
          dockercontainerRuns every process in the AMAP Docker image.
          singularitycontainerSame image via Singularity or Apptainer.
          standardcontainerNo container. Every tool must already be on PATH.
          slurmexecutorSubmits each process as a SLURM job.
          sgeexecutorSubmits to Sun Grid Engine or Univa Grid Engine.
          pbsexecutorSubmits to PBS or Torque.
          lsfexecutorSubmits to IBM Spectrum LSF.
          testdataSmall bundled dataset, for checking the installation.
          highmemresourcesRaises the ceilings to 48 CPUs, 256 GB, 120 h.
          
+
          ProfileKindWhat it does
          dockercontainerRuns every process in the AMAP Docker image.
          singularitycontainerSame image via Singularity or Apptainer.
          standardcontainerNo container. Every tool must already be on PATH.
          slurmexecutorSubmits each process as a SLURM job.
          sgeexecutorSubmits to Sun Grid Engine or Univa Grid Engine.
          pbsexecutorSubmits to PBS or Torque.
          lsfexecutorSubmits to IBM Spectrum LSF.
          testdataSmall bundled dataset, for checking the installation.
          highmemresourcesRaises the ceilings to 48ย CPUs, 256ย GB, 120ย h.
          
 
          Combine them with commas:
          
-
          nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv
          
-
          The order does not matter: each profile sets its own kind of thing. The ceilings on what any one task may ask for come from which profiles are on, not from their order: 48 CPUs, 256 GB and 120 h with highmem, 4 CPUs, 8 GB and 1 h with test, 48 CPUs, 128 GB and 72 h with an executor profile, and otherwise the machine's CPUs, 16 GB and 72 h. Without an executor profile everything runs on the local machine.
          
-
          On a laptop or workstation#
          
-
          nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
          
-
          CPUs are auto-detected and memory is capped at 16 GB. Raise both if the machine allows:
          
-
          nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --max_cpus 8 --max_memory 32.GB
          
-
          A handful of genomes is fine locally. Beyond about twenty, a cluster is worth the setup.
          
-
          On an HPC cluster#
          
-
          nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv
          
-
          Each process becomes its own scheduler job, with up to 200 running at once, so Bakta, Miniprot and eggNOG parallelise across all your samples. Defaults reach 48 CPUs and 128 GB per job, and every process carries a time limit so nothing sits in the queue forever.
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv
          
+
          The order does not matter: each profile sets its own kind of thing. The ceilings on what any one task may ask for come from which profiles are on, not from their order: 48ย CPUs, 256ย GB and 120ย h with highmem, 4ย CPUs, 8ย GB and 1ย h with test, 48ย CPUs, 128ย GB and 72ย h with an executor profile, and otherwise the machine's CPUs, 16ย GB and 72ย h. Without an executor profile everything runs on the local machine.
          
+
          On aย laptop or workstation
          #
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
          
+
          CPUs are auto-detected and memory is capped at 16ย GB. On a laptop under Docker Desktop, the VM's limit, often 8ย GB, is the one that counts: lower the cap to what docker info reports, and skip eggNOG, which asks for 50ย GB:
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --max_cpus 4 --max_memory 8.GB \
+    --skip_functional_annotation
          
+
          On a workstation with more, raise it:
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --max_cpus 16 --max_memory 64.GB
          
+
          What each step needs.
          
+
          A handful of genomes is fine locally. Past about twenty genomes, a cluster is worth the setup.
          
+
          On anย HPC cluster
          #
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv
          
+
          Each task becomes its own scheduler job, up to 200 at once (the checks, the versions and the parameters run on the launching node), so Bakta, miniprot and the correction run side by side across your samples. eggNOG is one job for all of them. Defaults reach 48ย CPUs and 128ย GB per job, and every process has a time limit, so a stuck job ends instead of holding its slot.
          
 
          To match your site's queues and accounting:
          
-
          nextflow run PathoGenOmics-Lab/AMAP \
-    -profile singularity,slurm \
-    --tsv samples.tsv \
-    --max_cpus 64 --max_memory 256.GB \
-    -process.queue long \
-    -process.clusterOptions '--account=mygroup'
          
-
          Deployment on HPC goes into the details that actually bite: shared filesystems, scratch, and where the eggNOG database should live.
          
-
          Large datasets
          Past a hundred genomes, use a scheduler. The eggNOG step in particular runs once over all samples at a time rather than once per sample, precisely so a shared filesystem is not hammered by hundreds of jobs reading the same Diamond database.
          
-
          Non-MTBC organisms#
          
-
          Lineage typing and Mycobrowser categories are specific to tuberculosis. Turn them off and bring your own reference:
          
-
          nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv \
-    --skip_typing --skip_mycobrowser \
-    --genus Escherichia --species "Escherichia coli" \
-    --ref_annotation my_reference.gbk --ref_genome my_reference.fasta
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP \
+    -profile singularity,slurm \
+    --tsv samples.tsv \
+    --max_cpus 64 --max_memory 256.GB \
+    -process.queue long \
+    -process.clusterOptions '--account=mygroup'
          
+
          HPC clusters goes into the details that actually bite: shared filesystems, scratch, and where the eggNOG database should live.
          
+
          TipLarge datasets
          Past about twenty genomes, use a scheduler. The eggNOG step in particular runs once over all the samples rather than once per sample, precisely so a shared filesystem is not hammered by hundreds of jobs reading the same Diamond database.
          
+
          Non-MTBC organisms
          #
          
+
          Lineage typing and Mycobrowser categories are specific to the Mycobacterium tuberculosis complex (MTBC). Turn them off and bring your own reference:
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv \
+    --skip_typing --skip_mycobrowser \
+    --genus Escherichia --species "Escherichia coli" \
+    --ref_annotation my_reference.gbk --ref_genome my_reference.fasta
          
 
          See custom reference.
          
-
          Resuming#
          
+
          Resuming
          #
          
 
          Nextflow caches every completed task, so an interrupted run picks up where it stopped:
          
-
          nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
-    --tsv samples.tsv -resume
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm \
+    --tsv samples.tsv -resume
          
 
          Keep the parameters identical. Changing an input invalidates that task and everything downstream, which is correct behaviour but does mean the work is redone. The cache lives in the work directory, so do not delete it between a run and its resume.
          
-
          Running only part of the pipeline#
          
-
          --skip_functional_annotation stops after the correction step. You get the corrected genes as a table with their proteins and sequences (*.AMAP.tsv, .faa, .fna) and the case tables, and skip eggNOG, duplications, the pangenome and the report. The GFF3 and GenBank files are written after eggNOG, so they are skipped too. Much faster when the corrected genes are all you are after:
          
-
          nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --skip_functional_annotation
          
-
          What a run produces#
          
-
          Beyond the results themselves, results/pipeline_info/ holds the execution report, the timeline, the DAG, a trace with per-process resource usage, software_versions.yml with the exact version of every tool, and params.yml with the parameters used. Those last two are what you want when writing up methods.
          
-
          Next steps#
          
+
          Running only part of the pipeline
          #
          
+
          --skip_functional_annotation stops after the correction step. You get the corrected genes as a table with their proteins and sequences (*.AMAP.tsv, .faa, .fna) and the case tables, and skip eggNOG, the duplications and the pangenome: no final_report.html; amap_report.html is still written, without its eggNOG sections. The GFF3 and GenBank files are written after eggNOG, so they are skipped too. Much faster when the corrected genes are all you are after:
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --skip_functional_annotation
          
+
          Pinning the pipeline
          #
          
+
          These pages describe AMAP as it is on main, which moves. Without -r, Nextflow runs the copy of AMAP it downloaded first and keeps it until you update it, so two machines can run different code under the same command. For an analysis you will publish, pin both the code and the image:
          
+
          Terminal
          $ nextflow run PathoGenOmics-Lab/AMAP \
+    -r 947a99dee49d2a2429e014296ac35ba08e911108 \
+    -profile docker --tsv samples.tsv \
+    -process.container 'paururo/amap@sha256:08bd17311ab323317ceecba6e5bc639a29f4c54c9d95f9a7c48ba80a9fc640ee'
          
+
          -r takes a branch, a tag or a commit. Give a commit as its full hash: Nextflowย 26.10 refuses a short one, with "Remote does not have 947a99d available for fetch", where 24.04 takes it. The image is pinned by its digest, which never moves, where a tag such as latest can; this one is the image behind the runs on these pages, and docker image inspect paururo/amap:latest --format '{{index .RepoDigests 0}}' prints the one you have.
          
+
          To move to the newest code, nextflow pull PathoGenOmics-Lab/AMAP. To see which code a run used, open pipeline_info/execution_report_*.html: it names the revision and its commit hash, as in revision main (commit hash 1a1b3959...).
          
+
          What aย run produces
          #
          
+
          Beyond the results themselves, results/pipeline_info/ holds the execution report, the timeline, the DAG, a trace with per-process resource usage, software_versions.yml with the exact version of every tool, and params.yml with the parameters used. Those last two are what you want when writing up methods.
          
+
          Next steps
          #
          
 
-
          Edit this page on GitHub
          
-
+
          Back to the top
          
+
          Edit this page on GitHub
          
+
 
          
-
 
 
-
 
 
 
diff --git a/docs/user-guide/running.md b/docs/user-guide/running.md
index 2291d6b..37fdbe0 100644
--- a/docs/user-guide/running.md
+++ b/docs/user-guide/running.md
@@ -43,16 +43,27 @@ the local machine.
 nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
 ```
 
-CPUs are auto-detected and memory is capped at 16 GB. Raise both if the machine
-allows:
+CPUs are auto-detected and memory is capped at 16 GB. On a laptop under Docker
+Desktop, the VM's limit, often 8 GB, is the one that counts: lower the cap to
+what `docker info` reports, and skip eggNOG, which asks for 50 GB:
 
 ```bash
 nextflow run PathoGenOmics-Lab/AMAP -profile docker \
-    --tsv samples.tsv --max_cpus 8 --max_memory 32.GB
+    --tsv samples.tsv --max_cpus 4 --max_memory 8.GB \
+    --skip_functional_annotation
 ```
 
-A handful of genomes is fine locally. Beyond about twenty, a cluster is worth
-the setup.
+On a workstation with more, raise it:
+
+```bash
+nextflow run PathoGenOmics-Lab/AMAP -profile docker \
+    --tsv samples.tsv --max_cpus 16 --max_memory 64.GB
+```
+
+[What each step needs](../getting-started/tutorial.md#the-machine).
+
+A handful of genomes is fine locally. Past about twenty genomes, a cluster is
+worth the setup.
 
 ## On an HPC cluster
 
@@ -60,10 +71,11 @@ the setup.
 nextflow run PathoGenOmics-Lab/AMAP -profile singularity,slurm --tsv samples.tsv
 ```
 
-Each process becomes its own scheduler job, with up to 200 running at once, so
-Bakta, Miniprot and eggNOG parallelise across all your samples. Defaults reach
-48 CPUs and 128 GB per job, and every process carries a time limit so nothing
-sits in the queue forever.
+Each task becomes its own scheduler job, up to 200 at once (the checks, the
+versions and the parameters run on the launching node), so Bakta, miniprot and
+the correction run side by side across your samples. eggNOG is one job for all
+of them. Defaults reach 48 CPUs and 128 GB per job, and every process has a
+time limit, so a stuck job ends instead of holding its slot.
 
 To match your site's queues and accounting:
 
@@ -76,20 +88,20 @@ nextflow run PathoGenOmics-Lab/AMAP \
     -process.clusterOptions '--account=mygroup'
 ```
 
-[Deployment on HPC](../deployment/hpc.md) goes into the details that actually
+[HPC clusters](../deployment/hpc.md) goes into the details that actually
 bite: shared filesystems, scratch, and where the eggNOG database should live.
 
 > [!TIP]
 > **Large datasets**
 >
-> Past a hundred genomes, use a scheduler. The eggNOG step in particular runs
-> once over all samples at a time rather than once per sample, precisely so a
-> shared filesystem is not hammered by hundreds of jobs reading the same
+> Past about twenty genomes, use a scheduler. The eggNOG step in particular
+> runs once over all the samples rather than once per sample, precisely
+> so a shared filesystem is not hammered by hundreds of jobs reading the same
 > Diamond database.
 
 ## Non-MTBC organisms
 
-Lineage typing and Mycobrowser categories are specific to tuberculosis. Turn
+Lineage typing and Mycobrowser categories are specific to the *Mycobacterium tuberculosis* complex (MTBC). Turn
 them off and bring your own reference:
 
 ```bash
@@ -121,9 +133,10 @@ and its resume.
 
 `--skip_functional_annotation` stops after the correction step. You get the
 corrected genes as a table with their proteins and sequences (`*.AMAP.tsv`,
-`.faa`, `.fna`) and the case tables, and skip eggNOG, duplications, the
-pangenome and the report. The GFF3 and GenBank files are written after eggNOG,
-so they are skipped too. Much faster when the corrected genes are all you are
+`.faa`, `.fna`) and the case tables, and skip eggNOG, the duplications and the
+pangenome: no `final_report.html`; `amap_report.html` is still written, without
+its eggNOG sections. The GFF3 and GenBank files are written after eggNOG, so
+they are skipped too. Much faster when the corrected genes are all you are
 after:
 
 ```bash
@@ -131,6 +144,32 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
     --tsv samples.tsv --skip_functional_annotation
 ```
 
+## Pinning the pipeline
+
+These pages describe AMAP as it is on `main`, which moves. Without `-r`,
+Nextflow runs the copy of AMAP it downloaded first and keeps it until you
+update it, so two machines can run different code under the same command. For
+an analysis you will publish, pin both the code and the image:
+
+```bash
+nextflow run PathoGenOmics-Lab/AMAP \
+    -r 947a99dee49d2a2429e014296ac35ba08e911108 \
+    -profile docker --tsv samples.tsv \
+    -process.container 'paururo/amap@sha256:08bd17311ab323317ceecba6e5bc639a29f4c54c9d95f9a7c48ba80a9fc640ee'
+```
+
+`-r` takes a branch, a tag or a commit. Give a commit as its full hash:
+Nextflow 26.10 refuses a short one, with "Remote does not have 947a99d
+available for fetch", where 24.04 takes it. The image is pinned by its digest,
+which never moves, where a tag such as `latest` can; this one is the image
+behind the runs on these pages, and
+`docker image inspect paururo/amap:latest --format '{{index .RepoDigests 0}}'`
+prints the one you have.
+
+To move to the newest code, `nextflow pull PathoGenOmics-Lab/AMAP`. To see
+which code a run used, open `pipeline_info/execution_report_*.html`: it names
+the revision and its commit hash, as in `revision main (commit hash 1a1b3959...)`.
+
 ## What a run produces
 
 Beyond the results themselves, `results/pipeline_info/` holds the execution
diff --git a/tests/benchmark/README.md b/tests/benchmark/README.md
index 4dcef9d..171918d 100644
--- a/tests/benchmark/README.md
+++ b/tests/benchmark/README.md
@@ -106,6 +106,12 @@ For each genome:
 - for the draft, how many proteins are the translation of their gene on the
   contig the table puts it on, and for the genes the cuts run through, how many
   of their pieces are annotated under the gene's name and marked partial
+- **Bakta alone**: exact and changed genes found again, for Bakta's own calls
+  of the same genome (the TSV it writes, kept with the inputs) instead of
+  AMAP's table. Bakta names a call after the reference where it annotated it
+  from the reference proteins AMAP gives it with `--proteins`: its DbXrefs then
+  hold `UserProtein:Rv0001`. That is the name scored, so the two are measured
+  against the same truth by the same rules
 
 A reference can give one gene several proteins from alternative start codons,
 all ending at its stop; K-12 does for cheA, infB, clpB and five more. AMAP
@@ -121,15 +127,21 @@ other, and whether a gene of the other run sits on the same reading frame there
 
 With the changes up to `changed-genes` and `contig-end-pieces` (October 2026):
 
-| Genome | exact | changed genes found | genes in the table | ORFs twice |
-| :--- | ---: | ---: | ---: | ---: |
-| H37Rv against itself | 4026 / 4026 | | 4031 | 0 |
-| H37Rv CP110619 | 3986 / 3986 | 50 / 50 | 4036 | 0 |
-| CDC1551 | 3097 / 3097 | 860 / 862 | 4014 | 2 |
-| M. bovis | 2391 / 2391 | 1515 / 1515 | 3947 | 3 |
-| CDC1551, 40 contigs | 3072 / 3072 | | 4050 | 2 |
-| E. coli K-12 against itself | 4286 / 4286 | | 4313 | 2 |
-| E. coli Sakai | 220 / 220 | 3363 / 3367 | 5334 | 11 |
+| Genome | exact | changed genes found | genes in the table | ORFs twice | Bakta alone: exact | Bakta alone: changed genes found |
+| :--- | ---: | ---: | ---: | ---: | ---: | ---: |
+| H37Rv against itself | 4026 / 4026 | | 4031 | 0 | 2905 | |
+| H37Rv CP110619 | 3986 / 3986 | 50 / 50 | 4036 | 0 | 2880 | 29 |
+| CDC1551 | 3097 / 3097 | 860 / 862 | 4014 | 2 | 2245 | 705 |
+| M. bovis | 2391 / 2391 | 1515 / 1515 | 3947 | 3 | 1742 | 1290 |
+| CDC1551, 40 contigs | 3072 / 3072 | | 4050 | 2 | 2231 | |
+| E. coli K-12 against itself | 4286 / 4286 | | 4313 | 2 | 3841 | |
+| E. coli Sakai | 220 / 220 | 3363 / 3367 | 5334 | 11 | 148 | 3202 |
+
+Bakta alone, scored by the same rules, finds 67 to 73% of the unchanged genes
+of each genome at exactly their coordinates and under their name (K-12, 90%),
+and AMAP all of them. What Bakta misses it misses by its coordinates: of the
+852 unchanged genes of CDC1551 it does not find, 836 have no call at their
+coordinates and 16 have one under another name.
 
 CDC1551 has one gene `deleted`, PPE50 (Rv3135). The six changed genes not
 found:
diff --git a/tests/benchmark/benchmark.py b/tests/benchmark/benchmark.py
index 9246df1..5b2c9d5 100755
--- a/tests/benchmark/benchmark.py
+++ b/tests/benchmark/benchmark.py
@@ -9,7 +9,8 @@
                   alignment, kept only where their sequence is identical
   truth-curated   add the curated H37Rv entries written in CP110619 coordinates
   draft           cut a complete genome into contigs, with a map back
-  score           how many truth genes an AMAP table has exactly, with the name
+  score           how many truth genes an AMAP table has exactly, with the name,
+                  and Bakta's own calls by the same rules (--bakta)
   compare         the genes of one run that another run does not have
 
 Everything here is independent of the algorithm being measured: the truth comes
@@ -234,6 +235,27 @@ def _load_amap(path):
     return a
 
 
+BAKTA_COLUMNS = ["id", "strand", "start", "end", "rvs", "pseudogene", "contig", "partial"]
+
+
+def _load_bakta(path):
+    """Bakta's own calls, the CDS rows of its TSV, in the shape of an AMAP
+    table, so they are scored by the same rules. A call is labelled with the
+    reference's name only where Bakta annotated it from the reference proteins
+    it was given with --proteins: its DbXrefs then hold UserProtein:NAME."""
+    rows = []
+    with open(path) as fh:
+        for line in fh:
+            if line.startswith("#") or not line.strip():
+                continue
+            f = (line.rstrip("\n").split("\t") + [""] * 9)[:9]
+            if f[1] != "cds":
+                continue
+            names = [x.strip()[len("UserProtein:"):] for x in f[8].split(",") if x.strip().startswith("UserProtein:")]
+            rows.append((f[5], f[4], int(f[2]), int(f[3]), ",".join(names), "no", f[0], ""))
+    return pd.DataFrame(rows, columns=BAKTA_COLUMNS)
+
+
 def _frame(start, end, strand):
     return (end if strand == "-" else start) % 3
 
@@ -256,29 +278,17 @@ def _doubles(df, contig_col):
     return n
 
 
-def score(args):
-    """Truth genes found at exactly their coordinates and strand, labelled with
-    their own name; plus the ORFs the table annotates twice.
-
-    A reference can give one gene several proteins from alternative start
-    codons, all ending at its stop (K-12 has cheA, infB, clpB and five more).
-    They are one gene, which AMAP annotates once, so they are scored as one
-    locus that counts as found if any of them is.
-    """
-    truth = pd.read_csv(args.truth, sep="\t")
-    changed = truth[truth.status == "changed"]
-    truth = truth[truth.status == "ok"]
-    amap = _load_amap(args.amap)
-    on_contigs = args.contig_map is not None
+def _exact(truth, table, on_contigs):
+    """Each truth locus as exact, wrong label or not exact in a table: the
+    names of each outcome, and how many loci there are."""
     if on_contigs:
-        truth = _truth_on_contigs(truth, args.contig_map)
         key = lambda r: (r.contig, int(r.start), int(r.end), r.strand)
         locus = lambda r: (r.name, r.contig, r.strand, r.start if r.strand == "-" else r.end)
     else:
         key = lambda r: (int(r.start), int(r.end), r.strand)
         locus = lambda r: (r.name, r.strand, r.start if r.strand == "-" else r.end)
     names = defaultdict(set)
-    for r in amap.itertuples():
+    for r in table.itertuples():
         names[key(r)] |= {_base(x.strip()) for x in r.rvs.split(",") if x.strip()}
     outcome = {}
     rank = {"exact": 2, "wrong label": 1, "not exact": 0}
@@ -296,7 +306,29 @@ def score(args):
     res = defaultdict(list)
     for got, name in outcome.values():
         res[got].append(name)
-    n = len(outcome)
+    return res, len(outcome)
+
+
+def score(args):
+    """Truth genes found at exactly their coordinates and strand, labelled with
+    their own name; plus the ORFs the table annotates twice.
+
+    A reference can give one gene several proteins from alternative start
+    codons, all ending at its stop (K-12 has cheA, infB, clpB and five more).
+    They are one gene, which AMAP annotates once, so they are scored as one
+    locus that counts as found if any of them is.
+
+    With --bakta, Bakta's own calls for the genome are scored by the same rules,
+    so the gain over Bakta alone is measured on the same truth.
+    """
+    truth = pd.read_csv(args.truth, sep="\t")
+    changed = truth[truth.status == "changed"]
+    truth = truth[truth.status == "ok"]
+    amap = _load_amap(args.amap)
+    on_contigs = args.contig_map is not None
+    if on_contigs:
+        truth = _truth_on_contigs(truth, args.contig_map)
+    res, n = _exact(truth, amap, on_contigs)
     isoforms = len(truth) - n
     doubles = _doubles(amap, "contig")
     line = {"genome": args.label, "truth": n, "exact": len(res["exact"]),
@@ -309,6 +341,16 @@ def score(args):
         line.update(_own_contig(amap, args.fasta, args.faa))
     if on_contigs:
         line.update(_cut_genes(args.truth, args.contig_map, amap))
+    # Last, so the columns before them keep their place in scores.tsv.
+    bakta_line = ""
+    if args.bakta:
+        bakta = _load_bakta(args.bakta)
+        bakta_res, _ = _exact(truth, bakta, on_contigs)
+        line["bakta_exact"] = len(bakta_res["exact"])
+        bakta_line = f"    Bakta alone: exact {line['bakta_exact']}/{n} ({100 * line['bakta_exact'] / max(n, 1):.2f}%)"
+        if "changed" in line:
+            line["bakta_changed_found"] = _changed_found(changed, bakta)["changed_found"]
+            bakta_line += f"  | changed genes found with their name {line['bakta_changed_found']}/{line['changed']}"
     print(f"{args.label:<14} exact {len(res['exact'])}/{n} ({100 * len(res['exact']) / max(n, 1):.2f}%)"
           f"  wrong label {len(res['wrong label'])}  not exact {len(res['not exact'])}"
           + (f" ({isoforms} isoforms counted with their gene)" if isoforms else "")
@@ -320,6 +362,8 @@ def score(args):
           + (f"  | genes cut by a contig end {line['cut']}: pieces with their name {line['cut_pieces_named']}"
              f" of {line['cut_pieces']}, {line['cut_pieces_partial']} of them marked partial"
              if "cut" in line else ""))
+    if bakta_line:
+        print(bakta_line)
     res["changed, not found"] = line.pop("changed_missed", [])
     for k in ("wrong label", "not exact", "changed, not found"):
         if res[k]:
@@ -468,6 +512,7 @@ def main():
     s.add_argument("--contig-map", help="score a draft: truth of the complete genome and its contig map")
     s.add_argument("--fasta", help="draft genome, to check each protein against its own contig")
     s.add_argument("--faa", help="the run's .AMAP.faa")
+    s.add_argument("--bakta", help="Bakta's TSV of the same genome, to score Bakta alone against the same truth")
     s.add_argument("--tsv", help="append the scores to this table")
     s.set_defaults(func=score)
 
diff --git a/tests/benchmark/benchmark.sh b/tests/benchmark/benchmark.sh
index 1b000f8..658f164 100755
--- a/tests/benchmark/benchmark.sh
+++ b/tests/benchmark/benchmark.sh
@@ -196,18 +196,25 @@ correct K-12 "$E" K12self "$E/custom_ref_ref_proteins.faa"
 correct Sakai "$E" Sakai "$E/custom_ref_ref_proteins.faa"
 
 # --- scores ------------------------------------------------------------------------
+# Bakta's own calls, cached with the inputs, are scored by the same rules: Bakta alone.
 echo
 echo "Run '$RUN' of $(cd "$REPO" && git rev-parse --short HEAD 2>/dev/null || echo 'this tree'):"
 S=$R/scores.tsv
-$BENCH score H37Rv-AL "$T/al.tsv" "$R/H37Rv-AL/L4_H37Rv.AMAP.tsv" --tsv "$S"
-$BENCH score H37Rv-CP "$T/cp.tsv" "$R/H37Rv-CP/L4_H37Rv.AMAP.tsv" --tsv "$S"
-$BENCH score CDC1551 "$T/cdc1551.tsv" "$R/CDC1551/L4_CDC1551.AMAP.tsv" --tsv "$S"
-$BENCH score M.bovis "$T/mbovis.tsv" "$R/M.bovis/La1_Mbovis.AMAP.tsv" --tsv "$S"
+$BENCH score H37Rv-AL "$T/al.tsv" "$R/H37Rv-AL/L4_H37Rv.AMAP.tsv" --tsv "$S" \
+    --bakta "$I/mtb_al/L4_H37Rv/L4_H37Rv.tsv"
+$BENCH score H37Rv-CP "$T/cp.tsv" "$R/H37Rv-CP/L4_H37Rv.AMAP.tsv" --tsv "$S" \
+    --bakta "$I/mtb_cp/L4_H37Rv/L4_H37Rv.tsv"
+$BENCH score CDC1551 "$T/cdc1551.tsv" "$R/CDC1551/L4_CDC1551.AMAP.tsv" --tsv "$S" \
+    --bakta "$I/mtb_cdc1551/L4_CDC1551/L4_CDC1551.tsv"
+$BENCH score M.bovis "$T/mbovis.tsv" "$R/M.bovis/La1_Mbovis.AMAP.tsv" --tsv "$S" \
+    --bakta "$I/mtb_mbovis/La1_Mbovis/La1_Mbovis.tsv"
 $BENCH score CDC1551-draft "$T/cdc1551.tsv" "$R/CDC1551-draft/L4_CDC1551d.AMAP.tsv" --tsv "$S" \
     --contig-map "$I/mtb_draft/contig_map.tsv" --fasta "$I/mtb_draft/L4_CDC1551d.fasta" \
-    --faa "$R/CDC1551-draft/L4_CDC1551d.AMAP.faa"
-$BENCH score K-12 "$T/k12.tsv" "$R/K-12/K12self.AMAP.tsv" --tsv "$S"
-$BENCH score Sakai "$T/sakai.tsv" "$R/Sakai/Sakai.AMAP.tsv" --tsv "$S"
+    --faa "$R/CDC1551-draft/L4_CDC1551d.AMAP.faa" --bakta "$I/mtb_draft/L4_CDC1551d/L4_CDC1551d.tsv"
+$BENCH score K-12 "$T/k12.tsv" "$R/K-12/K12self.AMAP.tsv" --tsv "$S" \
+    --bakta "$E/K12self/K12self.tsv"
+$BENCH score Sakai "$T/sakai.tsv" "$R/Sakai/Sakai.AMAP.tsv" --tsv "$S" \
+    --bakta "$E/Sakai/Sakai.tsv"
 echo "Scores in $S"
 
 if [ -n "$COMPARE" ]; then
diff --git a/tools/build_docs.py b/tools/build_docs.py
index c9f58d4..7187d04 100644
--- a/tools/build_docs.py
+++ b/tools/build_docs.py
@@ -25,10 +25,12 @@
 from __future__ import annotations
 
 import argparse
+import functools
 import html
 import json
 import re
 import sys
+import textwrap
 from pathlib import Path
 
 DOCS = Path("docs")
@@ -38,15 +40,15 @@
 # a .md file is built into a page; an .html path is a page written by hand and
 # only gets its place in the list.
 PAGES = [
-    ("Getting started", "getting-started/tutorial.md", "Tutorial", "From an empty folder to a corrected genome, with the numbers of a real run."),
     ("Getting started", "getting-started/installation.md", "Installation", "Nextflow, a container engine, and running without one."),
-    ("Getting started", "getting-started/quickstart.md", "Quickstart", "From an installed Nextflow to an open report."),
+    ("Getting started", "getting-started/quickstart.md", "Quickstart", "From an installed Nextflow to an open report: the test, a sample sheet, the check, the run and the results."),
+    ("Getting started", "getting-started/tutorial.md", "Tutorial", "One real genome, start to finish, with every number."),
     ("Getting started", "getting-started/inputs.md", "Preparing inputs", "The sample sheet, the genomes, and the checks run on them."),
     ("User guide", "user-guide/parameters.md", "Parameters", "Every option, its default, and when to change it."),
     ("User guide", "user-guide/running.md", "Running AMAP", "Profiles, schedulers and larger runs."),
     ("User guide", "user-guide/outputs.md", "Outputs", "Every folder and file, and what a healthy run looks like."),
-    ("User guide", "user-guide/report.md", "The report", "What each section of final_report.html shows."),
-    ("User guide", "user-guide/amap-report.md", "The interactive report", "amap_report.html, question by question, with or without eggNOG."),
+    ("User guide", "user-guide/report.md", "The full report", "Each section of the full report, and what it shows."),
+    ("User guide", "user-guide/amap-report.md", "The report by question", "Every question the report answers, with or without eggNOG."),
     ("User guide", "user-guide/custom-reference.md", "Custom reference", "Any organism, against a trusted annotation of your own."),
     ("How it works", "how-it-works.html", "Step by step, illustrated", "One gene at a time, from Bakta's call to what AMAP keeps."),
     ("How it works", "how-it-works/index.md", "Overview", "What goes into a decision, and the five outcomes."),
@@ -72,17 +74,26 @@
     " 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25"
     ".54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"
 )
-GH_SVG = f''
+# The icons carry a size of their own, so a page whose stylesheet does not
+# load (404.html opened from a clone) does not draw them a screen wide.
+GH_SVG = f''
 SUN_SVG = (
-    'E. coli K-12 against its own annotation. miniprot placed IbsC (b4665), 19 residues, only on its neighbour IbsD's locus; the exact search finds it here, between fau and serA, where Bakta called nothing."),
     "paralogs-1": case_figure("paralogs-1", "CDC1551 near 400 kb: Rv0336's own locus. Its paralog Rv0515 is placed here too, and that placement is dropped."),
     "paralogs-2": case_figure("paralogs-2", "Near 608 kb: Rv0515's own locus, where Rv0336's placement is dropped."),
-    "copies-1": case_figure("copies-1", "E. coli O157:H7 Sakai against K-12: the first of two whole copies of RhsD (b0497)."),
+    "copies-1": case_figure("copies-1", "E. coli O157:H7 Sakai against K-12: the first of two whole copies of RhsD (b0497)."),
     "copies-2": case_figure("copies-2", "The second copy, 1.4 Mb away, is a further copy and stays, as b0497_2."),
-    "frameshift": case_figure("frameshift", "M. bovis: Rv0590's frame runs on 67 residues past its protein, through Rv0590A, to the end of Bakta's one long call. It keeps its extent as a pseudogene, and Rv0590A is a gene of its own."),
+    "frameshift": case_figure("frameshift", "M. bovis: Rv0590's frame runs on 67 residues past its protein, through Rv0590A, to the end of Bakta's one long call. It keeps its extent as a pseudogene, and Rv0590A is a gene of its own."),
     "contig-end": case_figure("contig-end", "CDC1551 in 40 contigs: Rv1239c's frame reaches the end of contig_10 with no stop codon, and this piece of the gene, Rv1239c_2, is marked partial at its 3' end. The other piece, at the end of contig_9, is marked partial at its 5' end."),
     "contig-end-2": case_figure("contig-end-2", "The other piece of Rv1239c in the CDC1551 draft: its last 144 residues, placed by the second pass at the very end of contig_9. Bakta's short call over it is case 4, and the piece wins."),
     "case2": case_figure("case2", "Case 2 in CDC1551: dnaN, where Bakta and the aligned protein agree base for base."),
@@ -115,13 +126,13 @@ def case_figure(name: str, caption: str) -> str:
     "case4-split": case_figure("case4-split", "Multiple overlaps: two Bakta calls on Rv0007, which comes back as one gene."),
     "case4-longer": case_figure("case4-longer", "A much longer call: 3,303 bp past an alignment that stops short of its protein's end, so Bakta's call is kept, and it takes the protein's name, Rv2024c."),
     "case1-kept": case_figure("case1-kept", "Case 1 kept: four calls in a stretch CDC1551 has and H37Rv lacks."),
-    "case1-dropped": case_figure("case1-dropped", "Case 1 removed: a call that Bakta's known false positives cover end to end."),
+    "case1-dropped": case_figure("case1-dropped", "Case 1 dropped: a call that Bakta's known false positives cover end to end."),
     "missing": case_figure("missing", "A missing reference gene: Rv0031, put back between two of Bakta's calls."),
     "three-inputs": (
         '
          '
         "
          The three inputs over the first 12.4 kb of CDC1551: Bakta's calls, the H37Rv proteins miniprot placed, and "
-        "two of the calls Bakta makes on H37Rv itself that its curated annotation does not have, aligned here. Point at a gene "
-        "for its coordinates.
          "
+        "two of the calls Bakta makes on H37Rv itself that its curated annotation does not have, aligned here. "
+        'Point at a gene for its coordinates.'
     ),
     "outcomes": (
         '
          Bakta\'s call'
@@ -132,8 +143,8 @@ def case_figure(name: str, caption: str) -> str:
             f'{text}{more}'
             for key, tag, case, title, text, link, more in [
                 ("case2", "c2", "Case 2", "Perfect match", "Same start, end, strand and length as an aligned protein. Kept, tied to it.", "../how-it-works.html#agree", "Step through it"),
-                ("case3", "c3", "Case 3", "Imperfect match", "Same start, end and strand; the alignment's length differs. Kept, tied to the protein.", "correction-algorithm.html#1-classification", "The rule"),
-                ("case4-start", "c4", "Case 4", "Conflicting overlap", "Overlaps an aligned protein with other coordinates. The reference's win, unless the call is much longer.", "../how-it-works.html#late-start", "Step through it"),
+                ("case3", "c3", "Case 3", "Imperfect match", "Same start, end and strand; the alignment's length differs. Kept, tied to the protein.", "../how-it-works.html#same-ends", "Step through it"),
+                ("case4-start", "c4", "Case 4", "Conflicting overlap", "Overlaps an aligned protein with other coordinates. The reference wins, unless the call is much longer.", "../how-it-works.html#late-start", "Step through it"),
                 ("case1-kept", "c1", "Case 1", "Potentially novel", "Nothing aligns. Kept, unless Bakta is known to call it wrongly or the reference genome has its protein.", "../how-it-works.html#new", "Step through it"),
                 ("missing", "cm", "Missing", "Put back", "A reference protein aligns where Bakta called nothing. It goes in at the alignment.", "../how-it-works.html#missed", "Step through it"),
             ]
@@ -146,8 +157,8 @@ def case_figure(name: str, caption: str) -> str:
                                ("copies", "Two real copies of a gene")]
         )
         + "
          "
-        + "
          The five outcomes, each on real genes of CDC1551: the same windows as the case pages, "
-        "drawn small. Each card opens its story, step by step.
          "
+        + "
          The five outcomes, each on real genes of CDC1551: the same windows as the front page's cases and the "
+        "correction algorithm's figures, drawn small. Each card opens its story, step by step.
          "
     ),
     "dup-family": (
         '
          '
@@ -157,7 +168,7 @@ def case_figure(name: str, caption: str) -> str:
         'a protein outside the family'
         "
          CDC1551's corrected proteome against itself, run as GET_DUPLICATIONS runs it: six transposases make one "
         "family. Each line is a pair Diamond matched, with the larger of the two shares its identical residues cover; the "
-        "size of a protein is its length. Point at a protein or a line for its numbers.
          "
+        'size of a protein is its length. Point at a protein or a line for its numbers.'
     ),
     "dup-communities": (
         '
          '
@@ -167,8 +178,8 @@ def case_figure(name: str, caption: str) -> str:
         '
          '
         '
          a link in this genome'
         'a link only the other genome has
          '
-        "
          The same family in GET_ALLDUP, pooled with M. bovis AF2122/97's families: one community in each genome, "
-        "joined by their hits, and the row of homologous_communities.tsv they make. Each protein of M. bovis sits "
+        "
          The same family in GET_ALLDUP, pooled with M. bovis AF2122/97's families: one community in each genome, "
+        "joined by their hits, and the row of homologous_communities.tsv they make. Each protein of M. bovis sits "
         "where its closest match in CDC1551 does.
          "
     ),
     "metro": (
@@ -176,8 +187,16 @@ def case_figure(name: str, caption: str) -> str:
         ''
         ''
         ''
+        ''
         '
          Scroll sideways to follow the lines.
          '
-        '
          '
+        '
          '
+        "
          Every process of main.nf, in the order it runs them. The line under the panel lists what a run "
+        "with these switches writes.
          "
     ),
 }
 
@@ -193,6 +212,15 @@ def case_figure(name: str, caption: str) -> str:
 ]
 
 
+def count_label(value: int) -> str:
+    """A count printed in its bar. A long one (1,100 and up) says so, so a
+    phone's bar too narrow for it hides it (the key under the bar repeats
+    every count)."""
+    text = f"{value:,}"
+    wide = " cb-n-wide" if len(text) > 3 else ""
+    return f'{text}'
+
+
 def case_bars(genomes: list[tuple[str, dict[str, int]]], caption: str) -> str:
     """Bakta's calls of each genome by case, and the reference genes put back."""
     longest = max(sum(n.get(k, 0) for k, _, _ in CASE_PARTS) + n.get("rv_missing", 0) for _, n in genomes)
@@ -202,11 +230,11 @@ def case_bars(genomes: list[tuple[str, dict[str, int]]], caption: str) -> str:
         for key, label, cls in CASE_PARTS:
             if key in n:
                 width = 100 * n[key] / longest
-                number = f'{n[key]:,}' if width >= 8 else ""
+                number = count_label(n[key]) if width >= 8 else ""
                 segs.append(f'{number}')
         if "rv_missing" in n:
             width = 100 * n["rv_missing"] / longest
-            number = f'{n["rv_missing"]:,}' if width >= 8 else ""
+            number = count_label(n["rv_missing"]) if width >= 8 else ""
             segs.append(f'{number}')
         total = f'{n["total_cds"]:,} Bakta calls' if "total_cds" in n else ""
         values = "".join(
@@ -231,15 +259,20 @@ def results_map() -> str:
         ("main", "BAKTA/", "BAKTA_ANNOTATOR", ""),
         ("main", "CDS_CASES/", "TYPE_CASES", ""),
         ("main", "AMAP/", "FIX_CDS, then GET_OUTPUTS and GET_COUNT",
-         "with --skip_functional_annotation only FIX_CDS's *.AMAP.tsv, .faa and .fna are left"),
+         "With --skip_functional_annotation, only FIX_CDS's *.AMAP.tsv, .faa and .fna are written."),
         ("dup", "DUPLICATIONS/", "GET_DUPLICATIONS, GET_ALLDUP", "with --skip_functional_annotation"),
         ("pan", "PANGENE/", "PANGENE", "with --skip_functional_annotation"),
         ("main", "final_report.html", "FINAL_REPORT", "with --skip_functional_annotation"),
+        ("main", "amap_report.html", "AMAP_REPORT", ""),
         ("none", "pipeline_info/", "VALIDATE_INPUTS, SOFTWARE_VERSIONS, DUMP_PARAMS and Nextflow itself", ""),
     ]
     out = []
     for line, folder, by, when in rows:
-        gone = f'Left out {when}.' if when else 'Always written.'
+        # A note of its own starts with a capital; otherwise it says what leaves the folder out.
+        if when[:1].isupper():
+            gone = f'{when}'
+        else:
+            gone = f'Left out {when}.' if when else 'Always written.'
         out.append(
             f'
          {folder}'
             f'
          {by}{gone}
          '
@@ -300,7 +333,7 @@ def prepare_reference(numbers: dict[str, int]) -> str:
         '
          --ref_annotation + --ref_genome
          '
         f'
          {"".join(out)}
          '
         '
          The proteome is aligned onto every sample; the false positives rule out case 1 calls.
          '
-        "
          What happens to E. coli K-12 MG1655 given as the reference, once, before any sample is touched.
          "
+        "
          What happens to E. coli K-12 MG1655 given as the reference, once, before any sample is touched.
          "
     )
 
 
@@ -323,6 +356,35 @@ def slugify(text: str, seen: dict[str, int]) -> str:
     return slug
 
 
+# Anchors that links written before a heading was renamed still use: each
+# page's old id, and the heading it now belongs to. The changelog keeps
+# GitHub's anchor for every heading of CHANGELOG.md on its own (see
+# Renderer.heading), since links to those are written on GitHub.
+LEGACY_IDS = {
+    "getting-started/tutorial.md": {"where-to-go-next": "next-steps"},
+    "getting-started/quickstart.md": {
+        "1-write-a-sample-sheet": "2-write-a-sample-sheet",
+        "2-check-the-inputs": "3-check-the-inputs",
+        "3-run-it": "4-run-it",
+        "4-read-the-results": "5-read-the-results",
+        "5-resume-an-interrupted-run": "6-resume-an-interrupted-run",
+    },
+    "how-it-works/workflow.md": {"10-report": "10-the-full-report", "11-interactive-report": "11-the-report-by-question"},
+    "about/citation.md": {"citing-amap": "citation"},
+    "deployment/hpc.md": {"running-on-a-cluster": "hpc-clusters"},
+    "user-guide/custom-reference.md": {"using-your-own-reference": "custom-reference"},
+    "user-guide/report.md": {"the-report": "the-full-report-final_reporthtml", "kegg-pathways": "functional-annotation"},
+    "user-guide/amap-report.md": {"the-interactive-report": "the-report-by-question-amap_reporthtml"},
+    "how-it-works/index.md": {"how-it-works": "overview", "in-this-section": "next-steps"},
+}
+
+
+def anchor_alias(old: str, new: str) -> str:
+    """An old anchor that leads to a heading's current one; site.js puts the
+    current one in the address."""
+    return f''
+
+
 def plain(markdown_inline: str) -> str:
     """The text of an inline span, without its markup."""
     text = re.sub(r"`([^`]*)`", r"\1", markdown_inline)
@@ -335,6 +397,57 @@ def plain(markdown_inline: str) -> str:
     return re.sub(r"(?)\bK-12\b')
+
+
+def keep_names(fragment: str) -> str:
+    return WHOLE_NAMES.sub(r'\g<0>', fragment)
+
+
+def tie(text: str) -> str:
+    text = keep_names(text)
+    text = re.sub(rf"\b([A-Z])\. (?={SPECIES}\b)", rf"\1.{NBSP}", text)
+    text = re.sub(rf"(?<=\d) (?={UNITS}(?!\w))", NBSP, text)
+    text = re.sub(r"\b(case|Case|step|Step|Figure|version|Nextflow|Java) (?=\d)", rf"\1{NBSP}", text)
+    return re.sub(r"(?<=\d) to (?=\d)", f"{NBSP}to{NBSP}", text)
+
+
+def on_text(fragment: str, change) -> str:
+    """change() applied to the text of an HTML fragment: never to a tag, and
+    never inside code, which is copied as it reads."""
+    out, inside = [], 0
+    for part in re.split(r"(<[^>]+>)", fragment):
+        if part.startswith("<"):
+            tag = re.match(r"<(/?)(code|pre|kbd|script|style)\b", part, re.I)
+            if tag:
+                inside = max(inside + (-1 if tag.group(1) else 1), 0)
+            out.append(part)
+        else:
+            out.append(part if inside else change(part))
+    return "".join(out)
+
+
+def toc_label(markdown_inline: str) -> str:
+    """A heading's name in the page's contents, as HTML: its text, with each
+    span of code whole, so a file name never splits at a hyphen; a path may
+    still come apart after a /, to fit the sidebar."""
+    parts = re.split(r"`([^`]*)`", markdown_inline)
+    whole = lambda code: "".join(f'{html.escape(q)}' for q in re.findall(r"[^/]*/+|[^/]+", code))
+    return "".join(whole(p) if k % 2 else html.escape(plain(p)) for k, p in enumerate(parts))
+
+
 def strip_front_matter(lines: list[str]) -> list[str]:
     if lines and lines[0].strip() == "---":
         for i in range(1, len(lines)):
@@ -343,13 +456,330 @@ def strip_front_matter(lines: list[str]) -> list[str]:
     return lines
 
 
+# โ”€โ”€ Inline code โ”€โ”€
+#
+# A chip of code never splits inside a word. Up to SHORT_CODE characters it
+# does not wrap at all; a longer one wraps only at a space or after a /, _ or
+# ., so a path or a long file name comes apart where it reads in pieces. An
+# option name is one word: the longest, --skip_functional_annotation, fits a
+# phone's line whole. A piece of a few characters stays with the next one
+# ('/' or 'L4_' does not end a line alone) and an extension with its name
+# ('versions.tsv'). Each piece is a span that does not wrap, with a 
+# between pieces: the browser's own rules would also break after a hyphen,
+# '--|skip_typing'. A piece longer than LONG_TOKEN with nothing to break at, a
+# commit hash, may break anywhere, or it would run off a phone's screen.
+
+SHORT_CODE = 24
+SHORT_PIECE = 4
+
+
+def chip_pieces(token: str) -> list[str]:
+    """Where a chip of code without spaces may break: after a /, _ or ., with
+    a short piece kept on the next one and the extension on its name, and
+    nowhere in an option name."""
+    if token.startswith("-") and "/" not in token:
+        return [token]
+    pieces: list[str] = []
+    held = ""
+    for piece in re.findall(r"[^/_.]*[/_.]+|[^/_.]+", token):
+        held += piece
+        if len(held) > SHORT_PIECE:
+            pieces.append(held)
+            held = ""
+    if held:
+        pieces.append(held)
+    if len(pieces) > 1 and pieces[-2].endswith(".") and re.fullmatch(r"[A-Za-z0-9]+", pieces[-1]):
+        pieces[-2:] = [pieces[-2] + pieces[-1]]
+    return pieces
+
+
+def code_pieces(text: str) -> str:
+    out = []
+    for token in re.split(r"( +)", text):
+        if not token:
+            continue
+        if token.isspace():
+            out.append(token)
+            continue
+        pieces = chip_pieces(token)
+        out.append("".join(f'{html.escape(p)}' for p in pieces))
+    return "".join(out)
+
+
+def code_chip(text: str) -> str:
+    if len(text) <= SHORT_CODE:
+        return f'{html.escape(text)}'
+    return f"{code_pieces(text)}"
+
+
+def chips_in(fragment: str) -> str:
+    """The same treatment for the  of a figure written as HTML."""
+    return re.sub(r"([^<]*)", lambda m: code_chip(html.unescape(m.group(1))), fragment)
+
+
+# โ”€โ”€ Options โ”€โ”€
+#
+# Every --option of the parameters page has a row there with an id, and a
+# chip of code naming one anywhere else in the docs links to it.
+
+PARAMETERS = "user-guide/parameters.md"
+
+
+@functools.lru_cache(maxsize=None)
+def parameter_rows() -> list[dict]:
+    """The --option rows of the parameters page: flag, section, cells."""
+    rows, section, head = [], "", []
+    for line in (DOCS / PARAMETERS).read_text(encoding="utf-8").splitlines():
+        heading = re.match(r"^##\s+(.*)$", line)
+        if heading:
+            section = heading.group(1).strip()
+            continue
+        if not line.startswith("|"):
+            head = []
+            continue
+        cells = [c.strip() for c in re.split(r"(? set[str]:
+    return {row["flag"] for row in parameter_rows()}
+
+
+def switches() -> set[str]:
+    """Options that take no value: what follows one on a command line is not its value."""
+    return {row["flag"] for row in parameter_rows() if row["cells"].get("Type") == "boolean"} | {"--rm", "--help", "--version"}
+
+
+# Nextflow's own options that take a value.
+NF_VALUED = {"-profile", "-work-dir", "-c", "-r"}
+
+
+# โ”€โ”€ Code blocks โ”€โ”€
+
+SHELL = {"bash", "sh", "console", "shell"}
+TOKEN = re.compile(r"""(?:[^\s'"]|'[^']*'|"(?:\\.|[^"\\])*")+|\S+""")
+LONG_TOKEN = 36
+
+
+def open_quote(line: str, quote: str | None) -> str | None:
+    """The quote still open at the end of a line of shell, given the one open
+    at its start."""
+    i = 0
+    while i < len(line):
+        ch = line[i]
+        if quote is None:
+            if ch == "\\":
+                i += 2
+                continue
+            if ch == "#" and (i == 0 or line[i - 1].isspace()):
+                return None
+            if ch in "'\"":
+                quote = ch
+        elif ch == quote:
+            quote = None
+        elif ch == "\\" and quote == '"':
+            i += 2
+            continue
+        i += 1
+    return quote
+
+
+# A word of a command longer than this may come apart on a phone: a path
+# after a /, anything else (a commit hash, a quoted sentence) where it must.
+SHELL_LONG = 26
+
+
+def shell_token(token: str, kind: str) -> str:
+    """One word of a command, which never wraps inside itself unless it is
+    too long for a phone's line: an option name then breaks after an _ (its
+    dashes stay with its first word), a quoted word without spaces after a /
+    or a \t or \n escape (its closing quote stays with the piece before), a
+    path after a /, and anything else at its spaces or, failing those,
+    anywhere. A piece still too long for the line may break anywhere."""
+    pieces: list[str] = []
+    if len(token) > SHELL_LONG and token.startswith("-") and "_" in token and "/" not in token:
+        pieces = re.findall(r"[^_]*_+|[^_]+", token)
+    elif len(token) > SHELL_LONG and token.startswith(("'", '"')) and " " not in token and re.search(r"/|\\[tn]", token):
+        pieces = re.findall(r"[^/\\]*(?:/+|\\[tn])|[^/\\]+", token)
+        if len(pieces) > 1 and len(pieces[-1]) <= 2:
+            pieces[-2:] = [pieces[-2] + pieces[-1]]
+    if len(pieces) > 1 and "".join(pieces) == token:
+        inner = "".join(f'{html.escape(p)}' for p in pieces)
+        return f'{inner}'
+    if len(token) > SHELL_LONG and "/" in token and not token.startswith(("'", '"')):
+        # After each /, and in a file name too long for a line after _ or .
+        pieces = [q for p in re.findall(r"[^/]*/+|[^/]+", token)
+                  for q in (re.findall(r"[^_.]*[_.]+|[^_.]+", p) if len(p) > SHELL_LONG else [p])]
+        inner = "".join(f'{html.escape(p)}' for p in pieces)
+        return f'{inner}'
+    if len(token) > SHELL_LONG:
+        return f'{html.escape(token)}'
+    return f'{html.escape(token)}'
+
+
+def shell_line(line: str) -> str:
+    """A line of shell as words: flags (.f), their values (.v), other words
+    (.t) and a comment (.c). The spaces stay between the words, so a long
+    line wraps there and only there. A line's indent stays with its first
+    word, and a closing backslash with its last, so neither ends up on a line
+    of its own on a phone."""
+    units, at, tokens = [], 0, list(TOKEN.finditer(line))
+    k = 0
+    while k < len(tokens):
+        m = tokens[k]
+        gap = html.escape(line[at:m.start()])
+        word = m.group(0)
+        if word.startswith("#"):
+            units.append([gap, f'{html.escape(line[m.start():])}'])
+            at = len(line)
+            break
+        if re.match(r"^--?[A-Za-z]", word):
+            nxt = tokens[k + 1] if k + 1 < len(tokens) else None
+            valued = (
+                (word.startswith("--") and "=" not in word and word not in switches()) or word in NF_VALUED or word.startswith("-process.")
+            )
+            if valued and nxt and not re.match(r"^(-|#|\\$|\||&&|;|>)", nxt.group(0)):
+                value = nxt.group(0)
+                pair = shell_token(word, "f") + html.escape(line[m.end():nxt.start()]) + shell_token(value, "v")
+                units.append([gap, f'{pair}' if len(word) + len(value) < SHELL_LONG else pair])
+                at = nxt.end()
+                k += 2
+                continue
+            units.append([gap, shell_token(word, "f")])
+        else:
+            units.append([gap, shell_token(word, "t")])
+        at = m.end()
+        k += 1
+    if len(units) > 1 and tokens and tokens[-1].group(0) == "\\" and at == tokens[-1].end():
+        gap, slash = units.pop()
+        # Only the last word goes with it: of a flag and its value, the value.
+        body = units[-1][1]
+        last = re.search(r'(.*)$', body) if body.startswith('{tail}{gap}{slash}'
+        else:
+            units.append([gap, slash])
+    if units and units[0][0].strip() == "" and units[0][0]:
+        # The indent goes with the first word: of a flag and its value, the flag.
+        body = units[0][1]
+        first = re.match(r'([^<]*)(.+)$', body, re.S)
+        if first:
+            units[0] = ["", f'{units[0][0]}{first.group(1)}{first.group(2)}']
+        elif not re.match(r'{units[0][0]}{body}']
+    return "".join(gap + body for gap, body in units) + html.escape(line[at:])
+
+
+def shell_html(lines: list[str], prompts: bool) -> str:
+    """Each line of a shell block as a .ln, which wraps with a hanging
+    indent. A line that starts a command gets a $ prompt, which Copy leaves
+    out; a line that carries one on, after a backslash or inside a quote or
+    a loop, does not."""
+    out = []
+    carried = False
+    quote = None
+    depth = 0
+    for n, line in enumerate(lines):
+        stripped = line.strip()
+        indent = len(line) - len(line.lstrip(" "))
+        starts = prompts and stripped and not carried and quote is None and depth == 0 and not stripped.startswith("#")
+        prompt = '$ ' if starts else ""
+        # Inside a quote that spans lines is another program's text (an R
+        # call, say): its words still wrap only between them, uncoloured.
+        body = shell_line(line) if quote is None else re.sub(r"\S+", lambda m: shell_token(m.group(0), "t"), line)
+        style = f' style="--i:{indent}"' if indent else ""
+        end = "\n" if n + 1 < len(lines) else ""
+        out.append(f'{prompt}{body}{end}')
+        quote = open_quote(line, quote)
+        carried = stripped.endswith("\\")
+        if quote is None and not carried and stripped:
+            first = stripped.split()[0]
+            closes = re.search(r"\b(done|fi|esac)\b", stripped)
+            if first in ("for", "while", "until", "if", "case") and not closes:
+                depth += 1
+            elif first in ("done", "fi", "esac"):
+                depth = max(depth - 1, 0)
+    return "".join(out)
+
+
+def tsv_html(lines: list[str]) -> str:
+    """A tab-separated file, with each tab drawn. Copy takes the real tabs."""
+    return "\n".join(
+        '\t'.join(html.escape(cell) for cell in line.split("\t")) for line in lines
+    )
+
+
+CHANGELOG_EMOJI = re.compile("[\U0001F000-\U0001FAFFโ˜€-โžฟโฌ€-โฏฟ๏ธโ€]")
+MONTHS = ["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"]
+
+
+def version_heading(text: str) -> tuple[str, str, str]:
+    """A version of the changelog as the page names it: '[1.0.1] - 2026-07-25'
+    becomes ('1.0.1, 25 July 2026', 'v1-0-1', '1.0.1')."""
+    if re.fullmatch(r"\[?Unreleased\]?", text.strip(), re.I):
+        return "On main, not yet released", "unreleased", "On main"
+    m = re.match(r"^\[?([\d.]+)\]?(?:\s*-\s*(\d{4})(?:-(\d{2})-(\d{2}))?)?", text.strip())
+    if not m:
+        return text, slugify(text, {}), text
+    version, year, month, day = m.groups()
+    when = f"{int(day)} {MONTHS[int(month) - 1]} {year}" if day else (year or "")
+    return (f"{version}, {when}" if when else version), "v" + version.replace(".", "-"), version
+
+
+# The room a table has on a 360 px phone (326 px), with a margin for an
+# estimate that runs about 10% wide: measured, tables estimated at 341 fit
+# and those at 391 overflow. Then what a character takes there, code or
+# text, and a cell's padding.
+PHONE_TABLE = 360
+PHONE_CHAR = 7.8
+PHONE_CELL = 20
+
+
+def phone_width(head: list[str], body: list[list[str]]) -> float:
+    """About the narrowest a table can be: each column as wide as its widest
+    word or unbreakable chip, the first column's chips whole."""
+    columns = [0.0] * len(head)
+    for row in [head] + body:
+        for k, cell in enumerate(row[: len(head)]):
+            units = []
+            for m in re.finditer(r"`([^`]+)`", cell):
+                code = m.group(1).strip()
+                if k == 0 or len(code) <= SHORT_CODE:
+                    units.append(len(code) + 1.5)
+                else:
+                    units.append(max(len(p) for t in code.split() for p in chip_pieces(t)) + 1.5)
+            units += [len(w) for w in plain(re.sub(r"`[^`]+`", " ", cell)).split()]
+            columns[k] = max([columns[k]] + units)
+    return sum(c * PHONE_CHAR + PHONE_CELL for c in columns)
+
+
+def first_sentence(text: str, limit: int = 160) -> str:
+    m = re.match(r"(.+?[.:])(\s|$)", text)
+    cut = m.group(1) if m else text
+    return cut if len(cut) <= limit else cut[:limit].rsplit(" ", 1)[0] + "..."
+
+
 class Renderer:
     def __init__(self, source: str):
         self.source = source  # markdown path relative to docs/, e.g. user-guide/outputs.md
+        self.root = "../" * source.count("/")
+        self.changelog = source == CHANGELOG[0]
+        self.version: tuple[str, str] | None = None  # the changelog's version being read: (slug, label)
         self.seen: dict[str, int] = {}
-        self.headings: list[tuple[int, str, str]] = []  # (level, text, slug)
+        self.gh_seen: dict[str, int] = {}  # GitHub's anchors, for the changelog's aliases
+        self.legacy = {new: old for old, new in LEGACY_IDS.get(source, {}).items()}
+        self.headings: list[tuple[int, str, str]] = []  # (level, its contents entry as HTML, slug)
         self.sections: list[dict] = []  # for the search index
+        self.extra: list[dict] = []  # search entries of their own: callouts
         self.figures: list[str] = []
+        self.words = 0  # of running text, for the reading time
 
     # Links
 
@@ -373,20 +803,38 @@ def link(self, target: str) -> str:
 
     # Inline
 
-    def inline(self, text: str) -> str:
+    def inline(self, text: str, links: bool = True, whole: bool = False) -> str:
+        """An inline span as HTML. With links, a chip of code that names an
+        option of the parameters page links to its row there; whole keeps
+        a chip on one line, as the name of a table's row, unless it is longer
+        than a phone's line holds (a long file name then breaks after a _ or .)."""
         keep: list[str] = []
+        unlinked: dict[str, str] = {}
 
         def stash(fragment: str) -> str:
             keep.append(fragment)
             return f"\x00{len(keep) - 1}\x00"
 
-        text = re.sub(r"(`+)(.+?)\1", lambda m: stash(f"{html.escape(m.group(2).strip())}"), text)
+        def code(m: re.Match) -> str:
+            raw = m.group(2).strip()
+            chip = f'{html.escape(raw)}' if whole and len(raw) <= LONG_TOKEN else code_chip(raw)
+            flag = raw.split(" ")[0]
+            if links and flag in options() and self.source != PARAMETERS:
+                marker = stash(f'{chip}')
+                unlinked[marker] = chip
+                return marker
+            return stash(chip)
+
+        text = re.sub(r"(`+)(.+?)\1", code, text)
         text = re.sub(r"\\([\\`*_{}\[\]()#+\-.!|<>])", lambda m: stash(html.escape(m.group(1))), text)
         text = re.sub(r"<(https?://[^>\s]+)>", lambda m: stash(f'{html.escape(m.group(1))}'), text)
         text = INLINE_TAGS.sub(lambda m: stash(m.group(0)), text)
 
         def link(m: re.Match) -> str:
             label, target = m.group(1), m.group(2).strip().split(" ")[0]
+            # A link's own text keeps its chips plain: no link inside a link.
+            for marker, chip in unlinked.items():
+                label = label.replace(marker, stash(chip))
             return stash(f'') + label + stash("")
 
         text = re.sub(r"\[([^\]]+)\]\(([^)]+)\)", link, text)
@@ -394,6 +842,8 @@ def link(m: re.Match) -> str:
         text = re.sub(r"\*\*(.+?)\*\*", r"\1", text)
         text = re.sub(r"(?\1", text)
         text = re.sub(r"(?\1", text)
+        # The code is stashed, so only the words are tied.
+        text = tie(text)
         while "\x00" in text:
             text = re.sub(r"\x00(\d+)\x00", lambda m: keep[int(m.group(1))], text)
         return text
@@ -403,23 +853,29 @@ def link(m: re.Match) -> str:
     def blocks(self, lines: list[str]) -> str:
         out: list[str] = []
         i = 0
+        # A text block right after a shell block is what that command prints.
+        after_shell = False
         while i < len(lines):
             line = lines[i]
             stripped = line.strip()
             if not stripped:
                 i += 1
                 continue
+            shell = False
             figure = FIGURE_MARK.match(stripped)
             if figure:
                 name = figure.group(1)
                 if name in DERIVED:
-                    out.append(DERIVED[name](self, lines))
+                    out.append(on_text(chips_in(DERIVED[name](self, lines)), tie))
                 elif name in FIGURES:
-                    self.figures.append(name)
-                    out.append(FIGURES[name])
+                    # Only a drawing that figures.js draws needs it loaded.
+                    if FIGURE_HOOK.search(FIGURES[name]):
+                        self.figures.append(name)
+                    out.append(on_text(chips_in(FIGURES[name]), tie))
                 else:
                     raise SystemExit(f"docs/{self.source}: no figure called {name!r}")
                 i += 1
+                after_shell = False
                 continue
             if stripped.startswith("\n"
-        + body
-        + f'\n'
+        + (note or "\n")
+        + lead_ins(body)
+        # The search index is not loaded here: site.js fetches it the first
+        # time the reader goes to search.
         + f'\n'
         + (f'\n' if figures else "")
         + "".join(f'\n' for d in data)
         + (f'\n' if figures else "")
+        + tail
         + "\n\n"
     )
 
@@ -1286,75 +1994,418 @@ def source_lines(path: str) -> list[str]:
     return strip_front_matter((DOCS / path).read_text(encoding="utf-8").splitlines())
 
 
+def version_line(root: str) -> str:
+    """Which AMAP the pages describe: main, while it is ahead of the last release."""
+    ahead = any(name.lower() == "unreleased" and notes for name, notes in releases())
+    version = released()
+    if ahead:
+        return (f'
          Describes AMAP on main, ahead of {version}: '
+                f'what changed
          ')
+    return f'
          Describes AMAP {version}
          '
+
+
 def doc_page(position: int) -> tuple[Path, str, list[dict]]:
     section, path, title, description = PAGES[position]
     renderer = Renderer(path)
     content = renderer.blocks(source_lines(path))
+    if renderer.legacy:
+        raise SystemExit(f"tools/build_docs.py: {path} has no heading {sorted(renderer.legacy)} for LEGACY_IDS to lead to")
+    # The changelog's first line says what the standfirst already does, so
+    # the page and its search entry give the standfirst alone.
+    if renderer.changelog:
+        intro = "All notable changes to AMAP are documented in this file."
+        content = content.replace(f"
          {intro}
          \n", "", 1)
+        for s in renderer.sections[:1]:
+            s["text"] = [t for t in s["text"] if t != intro] or [description]
     root = "../" * path.count("/")
-    toc = "".join(
-        f'
        13. {html.escape(text)}
          14. '
-        for level, text, slug in renderer.headings
-        if level in (2, 3)
-    )
+    # The changelog's contents are its versions; its categories repeat in each.
+    levels = (2,) if renderer.changelog else (2, 3)
+    contents = page_contents(renderer.headings, levels)
+    h2s = [(text, slug) for level, text, slug in renderer.headings if level == 2]
+    # Under the h1: how long the page takes to read, and where there is no
+    # sidebar to list them (a phone), the contents themselves, folded.
+    minutes = max(1, round(renderer.words / 200))
+    # Under the h1, what the page is for: its line from PAGES, the one the
+    # Docs tab and a shared link show.
+    # The changelog is read a version at a time, not from end to end.
+    meta = "" if renderer.changelog else f'
          {minutes} min read
          \n'
+    under = f'
          {html.escape(description)}
          \n{meta}'
+    if h2s:
+        under += (
+            '
          On this page
            '
+            + "".join(f'
          • {text}
            • ' for text, slug in h2s)
+            + "
          \n"
+        )
+    end = content.find("")
+    if end >= 0:
+        content = content[: end + 5] + "\n" + under + content[end + 5 :]
+    # A long page ends with a way back up: one of many sections, or of many
+    # words (its tables' and blocks' too) in a few long ones.
+    long_page = len(h2s) >= 6 or len(html.unescape(re.sub(r"<[^>]+>", " ", content)).split()) >= 1500
+    to_top = '
          Back to the top
          \n' if long_page else ""
     prev_page = PAGES[position - 1] if position > 0 else None
     next_page = PAGES[position + 1] if position + 1 < len(PAGES) else None
-    pager = '
          ', front, re.S)
+        if not m:
+            raise SystemExit(f"tools/build_docs.py: docs/index.html has no section #{key} for the search index")
+        block = m.group(1)
+        title = re.search(r"]*>(.*?)", block, re.S)
+        kicker = re.search(r'
          (.*?)
          ', block, re.S)
+        name = tag_text(title.group(1)) if title else key
+        lead = re.search(r'
          (.*?)
          ', block, re.S)
+        text = tag_text(lead.group(1)) if lead else ""
+        if kicker:
+            text = text.replace(tag_text(kicker.group(1)), "", 1).strip()
+        out.append({"page": "Front page", "section": name, "url": f"index.html#{key}", "text": text, "kind": "section"})
+        if key == "faq":
+            for q in re.finditer(r"]*>\s*(.*?)(.*?)", block, re.S):
+                out.append({"page": "Front page", "section": tag_text(q.group(1)), "url": "index.html#faq",
+                            "text": tag_text(q.group(2)), "kind": "faq"})
+    return out
+
+
+def story_entries() -> list[dict]:
+    """Each story of how-it-works.html: its title and the words of its steps."""
+    how = (DOCS / "assets" / "how.js").read_text(encoding="utf-8")
+    nothing = dict(re.findall(r'^\s+(\w+): ("(?:\\.|[^"\\])*"),?$', how[how.index("var NOTHING"):how.index("var STORIES")], re.M))
+    page_title = next(title for _sec, path, title, _ in PAGES if path == "how-it-works.html")
+    out = [{"page": page_title, "section": "", "url": "how-it-works.html",
+            "text": next(line for _sec, path, _t, line in PAGES if path == "how-it-works.html"), "kind": "page"}]
+    stories = how[how.index("var STORIES"):]
+    starts = list(re.finditer(r'^\s+key: "([\w-]+)", group: "([^"]*)", label: "([^"]*)", tag: "([^"]*)"', stories, re.M))
+    if not starts:
+        raise SystemExit("tools/build_docs.py: no stories found in docs/assets/how.js for the search index")
+    for k, m in enumerate(starts):
+        chunk = stories[m.end(): starts[k + 1].start() if k + 1 < len(starts) else len(stories)]
+        steps = re.search(r"\btext: \[(.*?)\n\s+\]", chunk, re.S)
+        words = []
+        if steps:
+            for item in re.finditer(r'"(?:\\.|[^"\\])*"|NOTHING\[(\w+)\]|NOTHING\.(\w+)', steps.group(1)):
+                ref = item.group(1) or item.group(2)
+                words.append(json.loads(nothing[ref]) if ref else json.loads(item.group(0)))
+        key, group, label, tag = m.groups()
+        out.append({"page": page_title, "section": label, "url": f"how-it-works.html#{key}",
+                    "text": " ".join(words + [f"{tag}. {group}."]), "kind": "story"})
+    return out
+
+
+# The index goes out packed: each page's title and address once, each kind
+# once, and each entry as a row that points at them. The file unpacks itself
+# into the entries the search box reads, {page, section, url, text, code?,
+# kind}, as it loads; that keeps the whole of the docs under SEARCH_BUDGET.
+
+SEARCH_BUDGET = 230_000
+
+SEARCH_UNPACK = (
+    "window.AMAP_DOCS_INDEX = (function (d) {\n"
+    "  return d.rows.map(function (r) {\n"
+    "    var p = d.pages[r[0]];\n"
+    '    var e = { page: p[0], section: r[1], url: p[1] + (r[2] ? "#" + r[2] : ""), text: r[3] };\n'
+    "    if (r[5]) e.code = r[5];\n"
+    "    e.kind = d.kinds[r[4]];\n"
+    "    return e;\n"
+    "  });\n"
+    "})("
+)
+
+
+def pack_index(entries: list[dict]) -> dict:
+    pages: list[list[str]] = []
+    kinds: list[str] = []
+    rows = []
+    for e in entries:
+        base, _, fragment = e["url"].partition("#")
+        key = [e["page"], base]
+        if key not in pages:
+            pages.append(key)
+        if e["kind"] not in kinds:
+            kinds.append(e["kind"])
+        row = [pages.index(key), e["section"], fragment, e["text"], kinds.index(e["kind"])]
+        if e.get("code"):
+            row.append(e["code"])
+        rows.append(row)
+    return {"pages": pages, "kinds": kinds, "rows": rows}
+
+
+def unpack_index(packed: dict) -> list[dict]:
+    """What SEARCH_UNPACK makes of the packed index, for check_site.py."""
+    out = []
+    for r in packed["rows"]:
+        page, base = packed["pages"][r[0]]
+        e = {"page": page, "section": r[1], "url": base + ("#" + r[2] if r[2] else ""), "text": r[3]}
+        if len(r) > 5 and r[5]:
+            e["code"] = r[5]
+        e["kind"] = packed["kinds"][r[4]]
+        out.append(e)
+    return out
+
+
+def read_index(text: str) -> list[dict]:
+    """The entries of a search-index.js as build_docs.py writes it."""
+    return unpack_index(json.loads(text[text.index(SEARCH_UNPACK) + len(SEARCH_UNPACK):].rstrip().removesuffix(");")))
+
+
+def search_index_js(entries: list[dict]) -> str:
+    packed = pack_index(entries)
+    if unpack_index(packed) != entries:
+        raise SystemExit("tools/build_docs.py: the packed search index does not unpack to its entries")
+    return (
+        "/* Generated by tools/build_docs.py: the documentation, section by section, for the search box. */\n"
+        + SEARCH_UNPACK
+        + json.dumps(packed, ensure_ascii=False, separators=(",", ":"))
+        + ");\n"
+    )
 
 
 def build() -> dict[Path, str]:
@@ -1368,10 +2419,10 @@ def build() -> dict[Path, str]:
         entries += page_entries
     index_path, index_text = index_page()
     files[index_path] = index_text
-    files[SEARCH_INDEX] = (
-        "/* Generated by tools/build_docs.py: the documentation, section by section, for the search box. */\n"
-        "window.AMAP_DOCS_INDEX = " + json.dumps(entries, ensure_ascii=False, indent=0) + ";\n"
-    )
+    lost_path, lost_text = not_found_page()
+    files[lost_path] = lost_text
+    entries += front_page_entries() + story_entries()
+    files[SEARCH_INDEX] = search_index_js(entries)
     return files
 
 
diff --git a/tools/check_site.py b/tools/check_site.py
index 1396221..1628767 100644
--- a/tools/check_site.py
+++ b/tools/check_site.py
@@ -2,7 +2,7 @@
 """Check the website in docs/ against the pipeline it describes.
 
 The site is plain HTML with no build step, so nothing renders it and nothing
-fails when it drifts. This does, in nine parts:
+fails when it drifts. This does, in seventeen parts:
 
   links       every internal link and #anchor resolves (the stories how.js
               opens and the sections of the example report included), no page
@@ -26,9 +26,30 @@
   benchmark   the front page's scores are tests/benchmark's expected results;
   metro       the metro map's stations are main.nf's processes, in the order
               its workflow runs them;
+  words       no page says --skip_functional_annotation leaves no report
+              (amap_report.html is always written), or that a gene a contig
+              end cuts is lost (it comes back in pieces);
+  install     the front page, the installation page, the tutorial and
+              README.md install Nextflow with the same command;
+  citation    the citation page shows CITATION.cff as it is, and its APA line
+              and BibTeX give the version and date CITATION.cff does;
+  report      the example report, and the Rmd every final_report.html comes
+              from, load nothing from another host (MathJax included), and
+              the example has lang="en" and alt text on every image;
   readme      README.md's links land on a page and an anchor that exist, its
               pictures are in the repository, and its scores are the
               benchmark's;
+  stories     the story chips written into how-it-works.html are how.js's
+              stories, and no drawing hides a tab stop inside an image;
+  figures     a page that loads figures.js has something for it to draw;
+  search      the search index has the shape the search box reads, every
+              option has an entry that lands on its row, and the index stays
+              under its size budget;
+  front       the pages the command builder links to exist, the front
+              page's report frame and the picture standing in for it are
+              there, the picture light, and its rail lists Figure 1 and every
+              chapter in order, under the numbers the page gives them and by
+              words each chapter shows;
   dashes      no em-dashes.
 
 Run from the repository root, with what requirements-test.txt installs (the
@@ -677,7 +698,7 @@ def check_thresholds() -> list[str]:
                         f"{values.get('piece_edge_bp_pass')} bp in contig_end_pieces.py")
     # What the pages say each number is.
     said = {
-        "single_bp": [("index.html", r"longer by more than ([\d,]+) bp")],
+        "single_bp": [("index.html", r"longer by more than ([\d,]+)(?: | |\u00a0)bp")],
         "false_positive_pct": [("index.html", r"genome at (\d+)% identity")],
         "dup_cover_pct": [("how-it-works/duplications.html", r"(\d+)% or more"), ("how-it-works/duplications.html", r"under (\d+)%")],
         "dup_cover_pct_all": [("how-it-works/duplications.html", r"(\d+)% or more")],
@@ -767,10 +788,10 @@ def asks(process: str) -> tuple[int, int]:
         (running, r"(?P\d+) CPUs, (?P\d+) GB and (?P