Sourmash for relatively short sequences?

[Chiamh]

Hello,

Thanks for developing this tool! I know this isn't the intended use case, but is there a way to use sourmash search/gather for relatively short sequences (a few kb) against large databases? The sequences are so short that even scaled=100 seems inappropriate because there will be so few query hashes left.

[mr-eyes]

Hi!

Absolutely, you can query a large database with short seqs but you might need to make the scale=1

[Chiamh]

Thanks for the fast reply! I did consider smaller scales but both sourmash search and sourmash gather's help documentation says this:

--scaled FLOAT scaled value should be between 100 and 1e6

When I do sourmash sketch with -scaled=1 k=31, it completes but gives this warning:
WARNING: scaled value should be >= 100. Continuing anyway.

When I do sourmash gather...
sourmash gather short_sequence.fa.sig my_database.fa.sig --threshold-bp 2000 --dna -k 31 --scaled 1

WARNING: scaled value should be >= 100. Continuing anyway.
== This is sourmash version 4.3.0. ==
== Please cite Brown and Irber (2016), doi:10.21105/joss.00027. ==

selecting specified query k=31
loaded query: xxxx (k=31, DNA)
downsampling query from scaled=1 to 1
loaded 1923 signatures.

What does downsampling query from scaled=1 to 1 mean? Do I have to be concerned with these warning messages? Thanks again!

[mr-eyes]

Oh ok, at what scale did you build the database?

[Chiamh]

The same scale settings for everything. It was scaled=1

[mr-eyes]

Would you please retry with --threshold-bp 1 ?

[Chiamh]

Hello, I've tried it now with --threshold-bp 1 and it's the same warning:

sourmash gather representative_BGCs.id_TLL01_bin.12_c00123_opera_c...region001.fa.sig /home/ubuntu/volume4/iomics/antismash_mibig_db/mibig_all_bgcs.fa.sig --threshold-bp 1 --dna -k 31 --scaled 1
WARNING: scaled value should be >= 100. Continuing anyway.

== This is sourmash version 4.3.0. ==
== Please cite Brown and Irber (2016), doi:10.21105/joss.00027. ==
selecting specified query k=31
loaded query: representative_BGCs.id_TLL01_b... (k=31, DNA)
downsampling query from scaled=1 to 1

[mr-eyes]

Yes, the warning would still appear, but did the results change to something meaningful?

[Chiamh]

Hello,

Yes the final result for that specific example makes sense now. I will use these parameters for the rest of my searches then. Thanks!

[mr-eyes]

@Chiamh Glad it worked! Let us know if you needed more support :)

[mr-eyes]

What does downsampling query from scaled=1 to 1 mean? Do I have to be concerned with these warning messages? Thanks again!

Replying to this: If you built a database with a scale=1000 and then queried with a signature built with a scale of 100, the message would say "downsampling query from scaled=100 to 1000". The query scale must be >= the database scale.

[mr-eyes]

More information about --threshold-bp in #995 and https://sourmash.readthedocs.io/en/latest/command-line.html#:~:text=The%20command%20line%20option%20%2D%2Dthreshold%2Dbp%20sets%20the%20threshold%20below%20which%20matches%20are%20no%20longer%20reported%3B%20by%20default%2C%20this%20is%20set%20to%2050kb.%20see%20the%20Appendix%20in%20Classifying%20Signatures%20for%20details.

[ctb]

@Chiamh I removed the unnecessary output here, #1971 - thanks for noting it!

[ctb]

(it will be released in sourmash v4.4.0)