The get_ppl missed the last token of each iteration during multi-iter prefill

[lvhan028]

1. fix the missed last token in each prefill iteration
2. Assume the max memory footprint of logits is 2G, preventing OOM when batch_size * seq_len is too large

[lvhan028]

@grimoire can we remove the decode in class Engine?
The pipeline calls engine_instance's decode and get_ppl directly.

[grimoire]

can we remove the decode in class Engine?

Accept

[irexyc]

LGTM

the unit-test error is because the latest transformers changed their api and our logic for getting stop words invoked that function.

https://github.com/huggingface/transformers/blob/v4.44.2/src/transformers/tokenization_utils_base.py#L3526
https://github.com/huggingface/transformers/blob/main/src/transformers/tokenization_utils_base.py#L3514
https://huggingface.co/THUDM/chatglm-6b/blob/main/tokenization_chatglm.py#L349