Draft release notes for v2.0

[ctb]

git log --oneline v1.3..latest

Major changes for v2.0

With this v2.0 release, we've fixed several significant bugs, added thorough documentation, added several major new features and a variety of scripts, cleaned up the code base a fair bit, decreased memory usage by about 25%, and added many tests!

If you've been using spacegraphcats v1.x, the command line has changed - the main difference is that you now must specify at least one target before the config file. That is, instead of

python -m spacegraphcats <config file> <target> [<more targets>...]

you now run

python -m spacegraphcats <target> <config file> [<more targets>...]

We don't have a migration guide per se, but you can see the set of changes for the base pipeline in our paper here.

Please note that the file formats and locations have changed, so you'll need to rebuild any and all of your indexes.

---

Major new features:

• add paired read indexing and retrieval (add paired read indexing #344, require that some reads be paired, by default #356)
• improve BCALM input handling & sorting; reduce memory usage ~25% (improve BCALM input handling & sorting; reduce memory usage ~25% #310)

Bug fixes:

• Fix pendant removal bug potential bug during pendant removal #299 (improve BCALM input handling & sorting; reduce memory usage ~25% #310)
• Fix assignment bug in rdomset.domination_graph (fixes Vertex assignment bug in rdomset.domination_graph #390) (Fix assignment bug in rdomset.domination_graph (fixes #390) #394)

Major refactoring and new internal functionality:

• support randomizing unitig order via config parameter. ([MRG] support randomizing unitig order via config parameter. #391)
• handle duplicate hashes in input cDBG. (handle duplicate hashes in input cDBG. #375)
• add shadow annotations to support --shadow-prefix (add shadow annotations to support --shadow-prefix #365)
• Make default spacegraphcats interface use the click API (Make default spacegraphcats interface use the click API #353)
• use contigs sqlite DB, rather than contigs.fa.gz, for contig retrieval (use contigs sqlite DB, rather than contigs.fa.gz, for contig retrieval #345, More bcalm.unitigs.db/contigs/sqlite refactoring and cleanup #350, Attempt misc optimizations for extract_reads sqlite3 behavior. #361)
• remove python 2 support (chore: remove python 2 support #328)
• add default config via defaults.conf ([MRG] add defaults.conf; update docs to specify YAML instead of JSON #410)

Documentation enhancements:

• add documentation and mkdocs site for spacegraphcats ([MRG] add documentation and mkdocs site for spacegraphcats #392)

Refactoring and cleanup:

• relocate cDBG files to cDBG directory ([MRG] relocate cDBG files to cDBG directory #407)
• store ksize in MPHF index ([MRG] store ksize in MPHF index save files #406)
• properly close output files in bcalm_to_gxt ([MRG] properly close output files in bcalm_to_gxt #402)
• update rule make_contigs_kmer_index with contigs.sizes output ([MRG] update Snakefile with contigs.sizes in a rule output #401)
• update sourmash version requirements ([MRG] update sourmash version requirements to >=4.1,<5 #395)
• remove bcalm install test ([MRG] remove bcalm install test #396)
• rename bcalm_to_gxt2 to bcalm_to_gxt ([MRG] rename bcalm_to_gxt2 to bcalm_to_gxt #397)
• update sourmash versions in environment.yml (Update sourmash dep so that environment.yml installs from wheel again #367, [MRG] Update dib-lab software dependencies; fix warnings #385, [MRG] update sourmash to >=4 in environment.yml #389)
• Update test markers for pytest ([MRG] Update test markers for pytest #386)
• Update dib-lab software dependencies; fix warnings ([MRG] Update dib-lab software dependencies; fix warnings #385)
• update citation info ([MRG] update citation info in README from bioxriv to genome biology #381)
• use less yield, more memory (Attempt misc optimizations for extract_reads sqlite3 behavior. #361)
• adjust output messages (adjust output messages #359)
• move tests under tests/ (move tests under tests/ #343)
• add json2yaml script
• update tests to include buggy behavior
• add some shorter test files based on twofoo; automate tests
• ci: add flake to tests, format with black, and add precommit hooks (ci: add flake to tests, format with black, and add precommit hooks #326)
• chore: update click (chore: update click #327)
• ci: switch to github actions, fix some linting issues (ci: switch to github actions, fix some linting issues #323)
• Fix catlas checkpointing ([MRG] change self.dir to self.out_dir in catlas .py #412)
• add thread/cores parameter to bcalm in the Snakefile ([MRG] add thread/cores parameter to bcalm in the Snakefile #415)

[ctb]

editing here https://hackmd.io/jpNLHzl9SWqOuFFXu3N44A