LDA eta parameter auto-learning

[cscorley]

Adds auto-learning for the eta parameter on the LdaModel, a feature mentioned in Hoffman's Online LDA paper.

This PR is for a commit I cherry-picked from a fork by @joshua2ua. I've added some tests to ensure it works (=values actually change).

[tmylk]

Cool, tests do give something asymmetric.
To avoid code duplication, could update_alpha and update_eta call the same dimension-agnostic update_dir_prior function?

[cscorley]

@tmylk Not a bad idea, I've updated the PR with that.

[cscorley]

@tmylk While we're here, would it be a bad idea to use the same setup procedures for eta that we do in alpha, i.e., enforce a non-scalar value during init by creating the vector if one is given?

[tmylk]

Yes, that would make code much easier.
On 11 Oct 2015 16:43, "Christopher Corley" notifications@github.com wrote:

@tmylk https://github.com/tmylk While were here, would it be a bad idea
to use the same setup procedures for eta that we do in alpha, i.e., enforce
a non-scalar value during init?

—
Reply to this email directly or view it on GitHub
#479 (comment).

[cscorley]

@tmylk, sorry for the delay! Got around to fixing this up. Added a bunch of tests for the init behaviors of both alpha and eta.

[tmylk]

Looks pretty neat.

On a related note. Do you think that this should go on the wishlist: "support update_alpha in MultiCore mode"?
Requires changes to LdaModel to share alpha's numpy array between E-step worker threads. Multiprocessing does locks and shared memory.

[cscorley]

I would defer that to @ziky90.

[tmylk]

Regression test passed. (i.e. the tests that apply to both pre and post-PR functionality pass in both versions.)