* feature(zlx): Support async reset for envpool env manager

* fixbug(zlx): Add final_eval_reward in returned dict

* impala cnn encoder refactor.

* Minor change.

* test(wyh):add plot test code

* test(wyh):add plot test code

* test(wyh):add compare two different PICS codes

* test(wyh):add compare two different PICS

* test(wyh):compare two different PICS

* feature(nyz): add pure ppo policy gradient policy

* fix(nyz): fix flake8 style problem and remove redundant codes of continuous bc

* add infoNCE

* refine infonce test

* add dim

* fix style and pytest

* fix style and pytest

* add dqn_dim test

* fix style

* refine descriptions

* polish infoNCE & ST-DIM

* polish stdim & infonce loss

* fix style

* fix codecov

* fix import

* add readme

* update quick colab link

* add buffer description

* polish buffer api

* polish buffer api

* fix bugs in dmc2gym env.

* fix bugs of libOpenGL.so.0 cannot open shared object file.

* fix bugs in base env manager in subprocess mode when resetting and rendering dmc envs.

* fix dockerfile

* backward compatibity for tb video logging

Co-authored-by: ZHZisZZ <zhanhui@umich.edu>

* feature(pu): add board_gmaes env including tictactoe, gomoku, chess, go, atari

* style(pu): yapf format

* polish(pu): refactor

* polish(pu): polish as review

* polish(pu): move atari_muzero_env to dizoo/atari

* polish(pu):polish as review

* fix(pu): fix tictactoe expert action

* polish(pu): polish board_games env

* style(pu): yapf format

* draft steve

* remove breakpoint

* fix numerical instability

* polish

* polish STEVE

* change flatten_batch -> fold_batch

* change from VE to MVE in README.md

* speed up mbrl test

* Polish docstrings, create unsqueeze_repeat helper function

* fix assertion bug

* add trex example

* polish import

* merge main

* polish(lisong): add sqil cosine similarity of expert and agent grad

* polish(lisong): create SQILSACPolicy and add mujoco config

* feature(lisong): add sqil_sac example

* polish(lisong): use pendulum as demo env

* polish(lisong):polish sqil_continuous example

* polish(lisong): remove get config

* test(rjy):add discrete pendulum env

* test(rjy):fix discrete pendulum env

* test(rjy): change the parameter

* test(rjy): revise the pend_dqn config

* test(rjy): correct spelling mistakes

* test(rjy): modify the format

…384)

* dev(lwq) continuous examples: ddpg & td3

* dev(lwq) continuous examples: ddpg, d4pg & td3

* polish(lwq): polish VAE

* remove base VAE class

* Discard message sent by self in redis mq

* Set running flag of redis mq

* Rename nng finished to running

* Fix self._mq is none

* Turn deque into list

* add c51/qrdqn/iqn newpipeline example

* fix style