Add DOIs to proceedings and collections

[timj]

SPIE papers come with DOIs and ADS includes them in entries. It seems odd to me that we include DOIs for articles but not for proceedings papers that have them. SPIE papers are notoriously hard to parse since page numbers are usually missing from references.

[augustfly]

A quick comment bc I'd like to approve this PR as i completely agree with its conceptual goal.

Shouldn't the line read:

"format.pid output"

?

[timj]

Yes. Good grief. I'm sorry. Now I'll update with the actual change that works...

[timj]

Updated. Sorry about that. Trying not to commit the removal of trailing whitespace confused me a bit.

[augustfly]

Doing some testing. Goodness the ADS BibTeX is not that much better than the SPIE bibtex. I hope to approve this soon.

[augustfly]

As a note the test case is:

https://doi.org/10.1117/12.856347

or

https://ui.adsabs.harvard.edu/#abs/2010SPIE.7735E..0NB/abstract

[timj]

The SPIE version is awful since it uses commas to separate the authors and the pages entry is actually incorrect. ADS is better since for pages it properly uses the eid. Neither include the proceedings editors (ADS do try to fix that but the problem is upstream with SPIE).

[timj]

I added the DOI to incollection as well since it didn't seem it would do any harm. I did not explicitly test an incollection though.

[augustfly]

The ADS bibtex is much better wrt the authors but there is currently a mess (in my opinion; see the duplicated Volume/eid in the compiled version below) where ADS includes the Volume/eid are included in booktitle. As there really isn't a work around in a bst for variable bibtex, this doesn't appear related to the PR.

I want to check out a few more use cases if that is okay @timj

Bland-Hawthorn, J., Lawrence, J., Robertson, G., et al. 2010, in Proceedings of the SPIE, Volume 7735, id. 77350N (2010)., Vol. 7735, 77350N, doi: 10.1117/12.856347

[timj]

That's odd. For example, I'm looking at http://adsabs.harvard.edu/abs/2016SPIE.9911E..18A:

@INPROCEEDINGS{2016SPIE.9911E..18A,
   author = {{Angeli}, G.~Z. and {Xin}, B. and {Claver}, C. and {Cho}, M. and 
	{Dribusch}, C. and {Neill}, D. and {Peterson}, J. and {Sebag}, J. and 
	{Thomas}, S.},
    title = "{An integrated modeling framework for the Large Synoptic Survey Telescope (LSST)}",
booktitle = {Modeling, Systems Engineering, and Project Management for Astronomy VI},
     year = 2016,
   series = {\procspie},
   volume = 9911,
    month = aug,
      eid = {991118},
    pages = {991118},
      doi = {10.1117/12.2234078},
   adsurl = {http://adsabs.harvard.edu/abs/2016SPIE.9911E..18A},
  adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

and in my PDF I get:

Angeli, G. Z., Xin, B., Claver, C., et al. 2016, in Proc. SPIE, Vol. 9911, Modeling, Systems Engineering, and Project Management for Astronomy VI, 991118, doi: 10.1117/12.2234078

Which seems okay to me. I don't get any eid report.

[augustfly]

Right, sometimes ADS is including the Volume/eid/page into the booktitle as in these two (really quite random) examples:

@INPROCEEDINGS{2010SPIE.7735E..0NB,
       author = {{Bland-Hawthorn}, Joss and {Lawrence}, Jon and {Robertson}, Gordon and
        {Campbell}, Sam and {Pope}, Ben and {Betters}, Chris and {Leon-
        Saval}, Sergio and {Birks}, Tim and {Haynes}, Roger and
        {Cvetojevic}, Nick and {Jovanovic}, Nem},
        title = "{PIMMS: photonic integrated multimode microspectrograph}",
    booktitle = {Proceedings of the SPIE, Volume 7735, id. 77350N (2010).},
         year = 2010,
       volume = {7735},
        month = Jul,
          eid = {77350N},
        pages = {77350N},
          doi = {10.1117/12.856347},
       adsurl = {https://ui.adsabs.harvard.edu/#abs/2010SPIE.7735E..0NB},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

@INPROCEEDINGS{2013ASPC..475..383A,
       author = {{Allen}, A. and {Berriman}, B. and {Brunner}, R. and {Burger}, D. and
        {DuPrie}, K. and {Hanisch}, R.~J. and {Mann}, R. and {Mink}, J.
        and {Sandin}, C. and {Shortridge}, K. and {Teuben}, P.},
        title = "{Bring out your codes! Bring out your codes! (Increasing Software
        Visibility and Re-use)}",
     keywords = {Astrophysics - Instrumentation and Methods for Astrophysics, Computer
        Science - Digital Libraries, Computer Science - Software
        Engineering},
    booktitle = {Astronomical Data Analysis Software and Systems XXII. Proceedings of a
        Conference held at University of Illinois, Champaign, Illinois,
        USA 4-8 November 2012.  San Francisco: Astronomical Society of
        the Pacific, 2013, p.383},
         year = 2013,
       volume = {475},
        month = Oct,
        pages = {383},
       adsurl = {https://ui.adsabs.harvard.edu/#abs/2013ASPC..475..383A},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

[timj]

Wow. I've just looked and it seems to be that you are running into a new bumble bee bug. If I look at the same bibtex entry on bumble bee at https://ui.adsabs.harvard.edu/#abs/2016SPIE.9911E..18A/abstract I get:

@INPROCEEDINGS{2016SPIE.9911E..18A,
       author = {{Angeli}, G.~Z. and {Xin}, B. and {Claver}, C. and {Cho}, M. and
        {Dribusch}, C. and {Neill}, D. and {Peterson}, J. and {Sebag},
        J. and {Thomas}, S.},
        title = "{An integrated modeling framework for the Large Synoptic Survey Telescope
        (LSST)}",
    booktitle = {Proceedings of the SPIE, Volume 9911, id. 991118 20 pp. (2016).},
         year = 2016,
       volume = {9911},
        month = Aug,
          eid = {991118},
        pages = {991118},
          doi = {10.1117/12.2234078},
       adsurl = {https://ui.adsabs.harvard.edu/#abs/2016SPIE.9911E..18A},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

which is very wrong. @aaccomazzi any thoughts? Bumble bee was working fine a month back in February (I thought).

[timj]

And your ADASS example is fine on classic:

@INPROCEEDINGS{2013ASPC..475..383A,
   author = {{Allen}, A. and {Berriman}, B. and {Brunner}, R. and {Burger}, D. and 
	{DuPrie}, K. and {Hanisch}, R.~J. and {Mann}, R. and {Mink}, J. and 
	{Sandin}, C. and {Shortridge}, K. and {Teuben}, P.},
    title = "{Bring out your codes! Bring out your codes! (Increasing Software Visibility and Re-use)}",
booktitle = {Astronomical Data Analysis Software and Systems XXII},
     year = 2013,
   series = {Astronomical Society of the Pacific Conference Series},
   volume = 475,
archivePrefix = "arXiv",
   eprint = {1212.1915},
 primaryClass = "astro-ph.IM",
   editor = {{Friedel}, D.~N.},
    month = oct,
    pages = {383},
   adsurl = {http://adsabs.harvard.edu/abs/2013ASPC..475..383A},
  adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

[timj]

Oh, and bumblebee is not even including the series field. This is very wrong.

[aaccomazzi]

Looks like this is a problem with the generation of metadata fields for conference proceedings in our new system. I've opened an issue about it (see above). @timj the reason why these entries were "right" in bumblebee earlier was because we used to simply serve output from ADS classic, whereas now we natively create these entries from our new microservices. The usual bumps on the road...

[augustfly]

good times.