ENH(nextalign cli): show default values in --help usage statement

[AngieHinrichs]

Hi! I'm trying out nextalign on norovirus genomes (small ssRNA, ~7.5kb, but highly diverged), and most sequences are unalignable with nextalign's default settings (Unable to align: low seed matching rate. Details: number of seeds: 73, number of seed matches: 2, matching rate: 0.027, required matching rate: 0.300. Note that this sequence will not be included in the results.).

I'd like to try playing with the seed parameters. nextalign's --help statement describes the params but not their default values:

        --seed-length <SEED_LENGTH>
            k-mer length to determine approximate alignments between query and reference and
            determine the bandwidth of the banded alignment

        --mismatches-allowed <MISMATCHES_ALLOWED>
            Maximum number of mismatching nucleotides allowed for a seed to be considered a match

        --min-seeds <MIN_SEEDS>
            Minimum number of seeds to search for during nucleotide alignment. Relevant for short
            sequences. In long sequences, the number of seeds is determined by `--seed-spacing`

        --min-match-rate <MIN_MATCH_RATE>
            Minimum seed mathing rate (a ratio of seed matches to total number of attempted seeds)

        --seed-spacing <SEED_SPACING>
            Spacing between seeds during nucleotide alignment

It would be nice to know the default values as a starting point for exploring the parameter space. I guess I could figure out the seed length from the 'Unable to align' messages 🙂 but it would be very nice if the --help told them all. Thanks!

[corneliusroemer]

Thanks for the feedback, much appreciated 😊 We're very soon going to release version 3 which uses a much more robust seeding algorithm that should have no issues with Norovirus. We're using clap for the cli, I'm not sure whether it's easy to display defaults. I agree that it would be good to show what they are with help. Meanwhile you can find the defaults in parameters.rs, let me look it up.

…

On Mon, Sep 11, 2023, 23:10 Angie Hinrichs ***@***.***> wrote: Hi! I'm trying out nextalign on norovirus genomes (small ssRNA, ~7.5kb, but highly diverged), and most sequences are unalignable with nextalign's default settings (Unable to align: low seed matching rate. Details: number of seeds: 73, number of seed matches: 2, matching rate: 0.027, required matching rate: 0.300. Note that this sequence will not be included in the results.). I'd like to try playing with the seed parameters. nextalign's --help statement describes the params but not their default values: --seed-length <SEED_LENGTH> k-mer length to determine approximate alignments between query and reference and determine the bandwidth of the banded alignment --mismatches-allowed <MISMATCHES_ALLOWED> Maximum number of mismatching nucleotides allowed for a seed to be considered a match --min-seeds <MIN_SEEDS> Minimum number of seeds to search for during nucleotide alignment. Relevant for short sequences. In long sequences, the number of seeds is determined by `--seed-spacing` --min-match-rate <MIN_MATCH_RATE> Minimum seed mathing rate (a ratio of seed matches to total number of attempted seeds) --seed-spacing <SEED_SPACING> Spacing between seeds during nucleotide alignment It would be nice to know the default values as a starting point for exploring the parameter space. I guess I could figure out the seed length from the 'Unable to align' messages 🙂 but it would be very nice if the --help told them all. Thanks! — Reply to this email directly, view it on GitHub <#1253>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF77AQNSGAWFCT46A6CALCLXZ545VANCNFSM6AAAAAA4T5LZ2A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ivan-aksamentov]

The hardcoded defaults for v2 are here (branch v2):

nextclade/packages_rs/nextclade/src/align/params.rs

Lines 106 to 131 in 242d56f

	impl Default for AlignPairwiseParams {
	fn default() -> Self {
	Self {
	min_length: 100,
	penalty_gap_extend: 0,
	penalty_gap_open: 6,
	penalty_gap_open_in_frame: 7,
	penalty_gap_open_out_of_frame: 8,
	penalty_mismatch: 1,
	score_match: 3,
	max_indel: 400,
	seed_length: 21,
	min_seeds: 10,
	min_match_rate: 0.3,
	seed_spacing: 100,
	mismatches_allowed: 3,
	retry_reverse_complement: false,
	no_translate_past_stop: false,
	left_terminal_gaps_free: true,
	right_terminal_gaps_free: true,
	excess_bandwidth: 9,
	terminal_bandwidth: 50,
	gap_alignment_side: GapAlignmentSide::Right,
	}
	}
	}

For v3 (not stable, branch master) the hardcoded defaults are here:

nextclade/packages_rs/nextclade/src/align/params.rs

Lines 141 to 174 in 119cd4a

	impl Default for AlignPairwiseParams {
	fn default() -> Self {
	Self {
	min_length: 100,
	penalty_gap_extend: 0,
	penalty_gap_open: 6,
	penalty_gap_open_in_frame: 7,
	penalty_gap_open_out_of_frame: 8,
	penalty_mismatch: 1,
	score_match: 3,
	max_band_area: 500_000_000, // requires around 500Mb for paths, 2GB for the scores
	max_indel: 400, // obsolete
	seed_length: 21, // obsolete
	min_seeds: 10, // obsolete
	min_match_rate: 0.3, // obsolete
	seed_spacing: 100, // obsolete
	mismatches_allowed: 3, // obsolete
	retry_reverse_complement: false,
	no_translate_past_stop: false,
	left_terminal_gaps_free: true,
	right_terminal_gaps_free: true,
	gap_alignment_side: GapAlignmentSide::Right,
	excess_bandwidth: 9,
	terminal_bandwidth: 50,
	min_seed_cover: 0.33,
	kmer_length: 10, // Should not be much larger than 1/divergence of amino acids
	kmer_distance: 50, // Distance between successive kmers
	min_match_length: 40, // Experimentally determined, to keep off-target matches reasonably low
	allowed_mismatches: 8, // Ns count as mismatches
	window_size: 30,
	max_alignment_attempts: 3,
	}
	}
	}

There are 2 important changes to consider in the upcoming Nextclade v3:

• alignment algo is changed quite a bit, so the params will change
• Nextalign executable is removed. Instead, Nextclade will take over the same job. In the new dataset format most files will be optional (and the dataset is also optional, so individual input args can be used) - all this to emulate the interface of Nextalign and to facilitate incremental development of datasets.

Because we are removing Nextalign, it does not make sense to add params into its help text anymore, as we are not planning any more releases.

Regarding Nextclade: the datasets can (and do) override parameters (using virus_properties.json file for v2 and pathogen.json in the v3), because different viruses sometimes need some different tuning. So I think that the displayed hardcoded number might be inaccurate and misleading, depending on which dataset you are planning to run. But let me know if you think it makes sense to add hardcoded defaults to Nextclade v3 anyways.

In the meantime, one thing you can try is to add -v (--verbose) flag to the run command, and then the program should print the final values for this particular run, already taking into account values (in this order) in:

• dataset (if using Nextclade and if they are defined)
• CLI args (if an arg is provided)
• hardcoded defaults

UPD:

This statement is incorrect for v2:

already taking into account values (in this order) in

Nextclade/Nextalign v2 only print the CLI args, before merging-in the defaults, which is probably not very useful. This will change in v3.

[ivan-aksamentov]

If you want to try Nextclade v3:

You can download prebuilt binaries on GitHub Actions:

• Filter runs by branch "master": https://github.com/nextstrain/nextclade/actions?query=branch%3Amaster
• Go to "Artifacts" section
• Click on artifact named "out". It will be downloaded as "out.zip", which contains binaries for all platforms

Or you can build it from source, from master branch, using our dev guide:
https://github.com/nextstrain/nextclade/blob/master/docs/dev/developer-guide.md

But v3 is not released and not stable yet. It's a bit of a crazy land still, and things might break. In which case you can try a slightly earlier version in the list of GitHub Actions. When things calm down a bit, we'll probably release an alpha version, or a few.

We appreciate early testing and feedback!

[AngieHinrichs]

Thanks @ivan-aksamentov! I will give both a try. I see v3 can be run without a dataset if --input-ref is provided, great. 🚀