suppress log output for mallet LDA

[beyondeye]

I am running gensim LDA wrapper in a Google Cloud Run instance and I would like to suppress the logging during LDA build because it pollutes the logging output of the cloud run instance. It would be possible to add such an option to the gensim wrapper to mallet LDA?

[piskvorky]

IIRC we removed the Mallet wrapper from Gensim altogether in 4.0.0. CC @mpenkov .

If so, we can't add any new features to it, although you're free to copy the wrapper code from 3.8.3 and use it.

[beyondeye]

Thanks, I don't see any mention to it in the changelog. but ok thanks for your answer. For a small corpus I am getting better coherence with Mallet wrapper than built gensim lda. Should I try gensim nmf instead?

[piskvorky]

Yes, Gibbs sampling (~Mallet) gives more accurate results than variational methods (~Gensim).

NMF is another option for interpretable (non-negative) topics. Whether / which of these methods will work best for you depends on the task, and a million other factors like preprocessing, parameters etc.

[piskvorky]

Thanks, I don't see any mention to it in the changelog

You're right, that PR is still open = unreleased in 4.0.0beta: #2972

[beyondeye]

OK so I solved the issue: just for reference this is what I did:
in mallet distribution, I edited file mallet-2.0.8\class\cc\mallet\util\resources\logging.properties
and set .level= WARNING instead of .level= INFO.
By the way it is very unfortunate that you are dropping the ldamallet wrapper, but I guess it makes sense for you. I would suggest to keep the wrappers code in the code repository in some separate contrib package.
Thanks for your answers and explanations

[piskvorky]

Thanks for sharing your solution.

Yes, out of all the wrappers, the Mallet wrapper is the one people use the most. Perhaps we could keep this one… if there's someone who could commit to maintaining it.