Parallel compilation

[gribeill]

Work out a strategy for parallel compilation of sequence lists.

[dieris]

Check out this notebook, based on the parallel 1-q compilation by @matthewware

[gribeill]

I meant actual parallelism in the compiler, not just when compiling multiple files... I think this is doable, but it's major surgery so nothing urgent...

[matthewware]

The one option I can think of without major surgery is a total hack. Namely just split the sequences into N chucks on the front end, compile with N threads and manually stitch the compiled sequences .h5 file back together on the back end. I have faith we can engineer a more elegant solution.

[matthewware]

At any rate, it's a stretch goal for down the road as sequences get more complicated.

[dieris]

Oh, now I see what you mean. I guess I stopped reading at the title 🙈