TestWikiCorpus intermittent test hangs

[gojomo]

I've noticed a bunch of recent Travis-CI test 10-minute-hang-failures have been just after:

test_first_element (gensim.test.test_wikicorpus.TestWikiCorpus) ... 

One example: https://travis-ci.org/RaRe-Technologies/gensim/jobs/169348681

As with other intermittent test failures, this can really confuse evaluation of unrelated changes.

Perhaps something about its multiprocessing is flaky, and should be made less-aggressive?

[markroxor]

It led to a stalled build because of #956 as these lines were missing.
if __name__ == '__main__':
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.DEBUG)
unittest.main()
therefore the file test_wikicorpus.py did not return an "ok" when the build passed.

I tried fixing it in #957 . After this merge when I tested it on my VM this outputted "ok" unlike before where it's non-outputting nature led to "No output has been received in the last 10 minutes".

[gojomo]

I don't believe Travis-CI invokes these tests as individual scripts (where they'd be __main__). Also, when failing, often it's only one of the 4 Python versions we use – 2.7 in the cases I've noticed – whereas if this lack of __main__ handling was the real problem, I'd expect consistent failures.

[markroxor]

Indeed, you are right. Though my solution should have fixed it. Anyways I have lodged an issue with Travis already.

[gojomo]

If my conjecture about how Travis-CI runs the test is right, then your solution couldn't possibly have fixed it, because the added code would not be run/relevant.

[anmolgulati]

Weird though, but I removed a blank line in test_wikicorpus.py file and it works fine now. See PR #980 (Commit : here)

[gojomo]

@anmol01gulati Interesting! Unsure how that could help; perhaps some odd bit of corrupted cached files being used at Travis-CI?

[tmylk]

Removing an empty line has helped Travis before. We can't be doing this for every build, so issue is still open.