saveAsBed writes missing score values as '.' instead of '0'

[benwbooth]

When using ADAMContext.saveAsBed to write a GenomicRDD to a bed file, if the score is unset, it writes the '.' character as the score instead of '0'. Trying to convert that bed file to a bigBed file, I get:

pass1 - making usageList (17 chroms): 1 millis
Trailing characters parsing signed integer in field 5 line 1 of /data/seqdata/analysis/fakereads/S288C_reference_genome_R64-2-1_20150113/saccharomyces_cerevisiae_R64-2-1_20150113.genes.gff3.sorted.bed, got .

The '.' character is not a valid score in the BED format, unlike GFF. The score value must be set. A sensible default would be '0' or '1000'.

[benwbooth]

I just found another problem with ADAMCOntext.saveAsBed: Even if you explicitly set the score to 0 in the FeatureRDD, the score is not properly coerced into an Integer between 0 and 1000. The output bed file writes the score as '0.0'. Then bedToBigBed gives:

pass1 - making usageList (17 chroms): 3 millis
Trailing characters parsing signed integer in field 5 line 1 of /data/seqdata/analysis/fakereads/S288C_reference_genome_R64-2-1_20150113/saccharomyces_cerevisiae_R64-2-1_20150113.genes.gff3.sorted.bed, got 0.0

So there currently is no valid way to write to a BED file with ADAM. As a workaround I will have to manually modify the output file before running bedToBigBed.

[heuermh]

Thank you for submitting this issue, @benwbooth!

We follow the bedtools2 convention with regards to the score column in BED format:

score - The UCSC definition requires that a BED score range from 0 to 1000, inclusive. However, bedtools allows any string to be stored in this field in order to allow greater flexibility in annotation features. For example, strings allow scientific notation for p-values, mean enrichment values, etc. It should be noted that this flexibility could prevent such annotations from being correctly displayed on the UCSC browser.

Any string can be used. For example, 7.31E-05 (p-value), 0.33456 (mean enrichment value), “up”, “down”, etc.
This column is optional.

https://bedtools.readthedocs.io/en/latest/content/general-usage.html

That said, we could add an option to restrict the score field to the UCSC convention on save. What do you think?

[benwbooth]

I don't mind if the score value is stored as a string, but right now in org.bdgenomics.adam.sql.Feature it's stored as an Option[Double]:

https://static.javadoc.io/org.bdgenomics.adam/adam-core-spark2_2.11/0.24.0/index.html#org.bdgenomics.adam.sql.Feature

Is there any way the schema can be changed to store the score value as an Option[String]? That way I could format the score as an integer. Right now the score value always includes the decimal point because of how Doubles get converted to String, so the BED files are always incompatible with UCSC.

[heuermh]

No, we don't want to change the schema, as it represents the model across all feature formats.

There is a chart documenting the mappings here
https://github.com/heuermh/bdg-formats/blob/docs/docs/source/features.md

The issue is only with saving to text files in BED format, so we might include a method that given a minimum and maximum value, interpolates the Option[Double] score to an integer between 0 and 1000 (with 0 for missing values) and writes that out.

[benwbooth]

OK no problem, I was just going by the bedtools2 quote you posted that said the score was represented as a string:

Any string can be used. For example, 7.31E-05 (p-value), 0.33456 (mean enrichment value), “up”, “down”, etc.

Any solution that fixes UCSC compatibility is fine by me. Thanks!