Make legacy batching a global option

[XapaJIaMnu]

This should fix some tests failing @jerinphilip

[jerinphilip]

This commit fixes one test which was broken. Merge?

jerin@gna:/fs/bil0/jerin/code/marian-regression-tests$ MARIAN=../marian-dev/build CUDA_VISIBLE_DEVICES=0,1 ./run_mrt.sh tests/scorer/scores/test_scores_cpu.sh
[02/06/2021 23:15:06] Running on gna as process 22501
[02/06/2021 23:15:06] Using Marian binary: /mnt/bil0/jerin/code/marian-dev/build/marian
[02/06/2021 23:15:06] Version: v1.9.56 db371bb 2021-02-06 22:36:02 +0000
[02/06/2021 23:15:07] Build type: Release
[02/06/2021 23:15:07] Using compiler: /usr/bin/c++
[02/06/2021 23:15:07] Using MKL: COMPILE_CPU=on
[02/06/2021 23:15:07] Using CUDNN:
[02/06/2021 23:15:07] Using SentencePiece: USE_SENTENCEPIECE=ON
[02/06/2021 23:15:07] Using FBGEMM:
[02/06/2021 23:15:07] Unit tests: COMPILE_TESTS=ON
[02/06/2021 23:15:07] Using CUDA visible devices: 0,1
[02/06/2021 23:15:07] Using number of GPU devices: 2
[02/06/2021 23:15:07] Using time out: 5m
[02/06/2021 23:15:07] Checking directory: tests/scorer/scores
[02/06/2021 23:15:07] Running setup script
[02/06/2021 23:15:07] Running tests/scorer/scores/test_scores_cpu.sh ... OK
[02/06/2021 23:15:46] Test took 00:00:38.529s
---------------------
Ran 1 tests in 00:00:38.985s, 1 passed, 0 skipped, 0 failed