Parsoid?

[legoktm]

Have you looked into the output that Parsoid produces? It converts wikitext into HTML/RDFa (and back!) and is expected to eventually replace the current MediaWiki parser.

An example is: http://parsoid-lb.eqiad.wikimedia.org/enwiki/Main_Page?oldid=598252063 (API docs at https://www.mediawiki.org/wiki/Parsoid/API.

[spencermountain]

hi Kunal, yes thanks. Parsoid is certainly profoundly cool, and does prompt me to have less angst. ;)
My hope is to have structured query-able data from a wp dump, and it's totally easier to pick-out data in html than in markup, and especially getting plaintext, but it does require some kind of headless dom thing, on 4m articles, and honestly there's more formatting stuff to remove in the html than in the markup itself.
short answer, I thought about it, then drank too much coffee.

Do you know if there's some kind of intermediate step before parsoid makes its html?
Like, some kind of representation that is more hackable?
I would be very excited if so
thanks for the heads-up, too.

[spencermountain]

added a section to the readme, thanks