Compress test vasprun.xml files #3648
Conversation
DanielYang59
requested review from
shyuep,
janosh and
mkhorton
as code owners
|
Meanwhile I noticed there are |
Remove them if they are not used anymore |
There was a problem hiding this comment.
excellent work @DanielYang59! this has been on my todo list for a very long time. 🙏
in retrospect, i should have proceeded in smaller chunks like this PR rather trying to do everything at once in #3132.
i know this must have taken some time, so much appreciated!
please do! that would be nice |
No problem at all @janosh. This cannot have been done so easily without the foundation laid in #3132. Yes I guess chopping up tasks into small pieces make everything much easier (easier to work on, easier to review and easier to track and revert in case anything goes wrong). I just notice that |
no need, please keep making PRs to |
Compressed (
gzip) test vasprun.xml files.Summary:
Total vasprun.xml file size reduced by 78.67% (361 MB -> 77 MB), with test runtime largely unaffected (increased by 1.05% but could be random error)
Before compression:
Total vasprun.xml file size:
Test runtime:
........................................................................... [ 25%] ........................................................................... [ 51%] ........................................................................... [ 77%] ................................................................. [100%] ============================== slowest 30 durations =============================== 7.53s call tests/io/vasp/test_outputs.py::TestVasprun::test_runtype 6.71s call tests/io/vasp/test_optics.py::TestVasprun::test_optics 4.45s call tests/io/vasp/test_outputs.py::TestOutcar::test_stopped 4.44s call tests/io/vasp/test_outputs.py::TestOutcar::test_stopped_old 4.41s call tests/io/vasp/test_sets.py::TestMPStaticSet::test_grid_size_from_struct 2.93s call tests/io/vasp/test_outputs.py::TestAeccars::test_read_write_file 2.59s call tests/io/vasp/test_outputs.py::TestVasprun::test_standard 2.38s call tests/io/vasp/test_outputs.py::TestOutcar::test_avg_core_poten 2.35s call tests/io/vasp/test_outputs.py::TestVasprun::test_projected_magnetisation 2.24s call tests/io/vasp/test_outputs.py::TestVasprun::test_dfpt_ionic 2.14s call tests/io/vasp/test_outputs.py::TestOutcar::test_read_piezo_tensor 1.75s call tests/io/vasp/test_outputs.py::TestOutcar::test_init 1.53s call tests/io/vasp/test_outputs.py::TestVasprun::test_kpointset_electronvelocities 1.53s call tests/io/vasp/test_outputs.py::TestVasprun::test_update_potcar 1.35s call tests/io/vasp/test_outputs.py::TestVasprun::test_nonlmn 1.26s call tests/io/vasp/test_outputs.py::TestVasprun::test_eigenvalue_band_properties_separate_spins 1.25s call tests/io/vasp/test_outputs.py::TestChgcar::test_init 0.97s call tests/io/vasp/test_outputs.py::TestVasprun::test_smart_efermi 0.88s call tests/io/vasp/test_outputs.py::TestOutcar::test_nplwvs 0.86s call tests/io/vasp/test_outputs.py::TestOutcar::test_drift 0.85s call tests/io/vasp/test_outputs.py::TestVasprun::test_energies 0.83s call tests/io/vasp/test_outputs.py::TestChgcar::test_soc_chgcar 0.66s call tests/io/vasp/test_sets.py::TestMPAbsorptionSet::test_rpa 0.62s call tests/io/vasp/test_outputs.py::TestVasprun::test_as_dict 0.58s call tests/io/vasp/test_outputs.py::TestVasprun::test_get_band_structure 0.56s call tests/io/vasp/test_outputs.py::TestOutcar::test_read_table_pattern 0.55s call tests/io/vasp/test_outputs.py::TestOutcar::test_electrostatic_potential 0.51s call tests/io/vasp/test_outputs.py::TestVasprun::test_search_for_potcar 0.50s setup tests/io/vasp/test_outputs.py::TestChgcar::test_add 0.44s call tests/io/vasp/test_outputs.py::TestVasprun::test_float_overflow 290 passed in 77.92s (0:01:17)After compression:
Total vasprun.xml file size:
Test runtime:
........................................................................... [ 25%] ........................................................................... [ 51%] ........................................................................... [ 77%] ................................................................. [100%] ============================== slowest 30 durations =============================== 8.01s call tests/io/vasp/test_outputs.py::TestVasprun::test_runtype 6.54s call tests/io/vasp/test_optics.py::TestVasprun::test_optics 4.70s call tests/io/vasp/test_outputs.py::TestOutcar::test_stopped 4.61s call tests/io/vasp/test_outputs.py::TestOutcar::test_stopped_old 4.24s call tests/io/vasp/test_sets.py::TestMPStaticSet::test_grid_size_from_struct 2.73s call tests/io/vasp/test_outputs.py::TestAeccars::test_read_write_file 2.68s call tests/io/vasp/test_outputs.py::TestVasprun::test_standard 2.44s call tests/io/vasp/test_outputs.py::TestVasprun::test_dfpt_ionic 2.38s call tests/io/vasp/test_outputs.py::TestVasprun::test_projected_magnetisation 2.35s call tests/io/vasp/test_outputs.py::TestOutcar::test_avg_core_poten 2.14s call tests/io/vasp/test_outputs.py::TestOutcar::test_read_piezo_tensor 1.72s call tests/io/vasp/test_outputs.py::TestOutcar::test_init 1.57s call tests/io/vasp/test_outputs.py::TestVasprun::test_update_potcar 1.52s call tests/io/vasp/test_outputs.py::TestVasprun::test_kpointset_electronvelocities 1.32s call tests/io/vasp/test_outputs.py::TestVasprun::test_eigenvalue_band_properties_separate_spins 1.30s call tests/io/vasp/test_outputs.py::TestVasprun::test_nonlmn 1.17s call tests/io/vasp/test_outputs.py::TestChgcar::test_init 1.01s call tests/io/vasp/test_outputs.py::TestVasprun::test_smart_efermi 0.94s call tests/io/vasp/test_outputs.py::TestOutcar::test_nplwvs 0.88s call tests/io/vasp/test_outputs.py::TestOutcar::test_drift 0.86s call tests/io/vasp/test_outputs.py::TestChgcar::test_soc_chgcar 0.85s call tests/io/vasp/test_outputs.py::TestVasprun::test_energies 0.64s call tests/io/vasp/test_sets.py::TestMPAbsorptionSet::test_rpa 0.64s call tests/io/vasp/test_outputs.py::TestVasprun::test_as_dict 0.55s call tests/io/vasp/test_outputs.py::TestOutcar::test_read_table_pattern 0.54s call tests/io/vasp/test_outputs.py::TestVasprun::test_search_for_potcar 0.52s call tests/io/vasp/test_outputs.py::TestOutcar::test_electrostatic_potential 0.51s setup tests/io/vasp/test_outputs.py::TestChgcar::test_add 0.47s call tests/io/vasp/test_outputs.py::TestVasprun::test_get_band_structure 0.43s call tests/io/vasp/test_outputs.py::TestVasprun::test_float_overflow 290 passed in 78.74s (0:01:18)Project-wise
pytestrunExpand
Apparently I have many missing dependencies...
Follow-up work
I would go ahead to clean up (notice all VASP test files seem to be placed flat inside the
filesdir, they might deserve their ownvaspdir) and compress some other VASP test files (CHGCAR*, OUTCAR*) later. Would open a separate PR for that.