NYT downloader broken due to removal of .puz files

[jpd236]

FYI, the NYT downloader is no longer functional as of today as .puz files have been removed from the site (see https://www.nytimes.com/2021/08/02/crosswords/nyt-games-no-longer-available-on-across-lite-as-of-aug-9.html).

At least at the moment, the puzzle data is embedded on the puzzle HTML pages as a bit of compressed JSON, so it can be scraped, though it needs another layer to convert it. And there are some features like image overlays on the grid that aren't supported by any format XWord currently supports, to my knowledge (though perhaps they could be handled with a proprietary ipuz extension or something along those lines).

If it helps, here's a reference implementation of a parser from the raw HTML: https://github.com/jpd236/kotwords/blob/master/src/commonMain/kotlin/com/jeffpdavidson/kotwords/formats/NewYorkTimes.kt

[mrichards42]

That's unfortunate. I was under the impression that the NYT downloader stopped working a long time ago b/c they changed their login form. Do you know if it was actually working before this?

[jpd236]

Ah - I hadn't checked it myself, was just passing along the info. You're right that the login form has changed, and #54 (comment) does note that it wasn't fixed as part of the last sweep.

If you ever do get the login working again, then this would also likely need to be fixed. That said, at least for now, it appears the .puz files are still being shadow posted at a hidden URL (https://nytimes.com/svc/crosswords/v2/puzzle/daily-YYYY-MM-DD.puz), so maybe this won't be necessary for a while. Who knows :)