DView Loading Data Extremely Slowly

[anchapin]

I'm using Windows 10 Enterprise Edition, 64-bit with Intel Core i7-6920HQ, 32 GB of RAM, both OpenStudio 2.5.0 and 2.6.0 and when I run a simulation and click the "Open DView for Detailed Reports" button it takes a really long time to load DView (for example, right now I've been waiting about 21 minutes and still don't see the DView window). The eplusout.sql file is 47.6 MB.

Note that if I try to delete eplusout.sql I get a File In Use message that says that the file is open in DView.exe. However, I can't see the DView window.

[nickdiorio]

Hi @anchapin , thanks for the report. @macumber, have you seen any other issues with eplusout.sql files loading slowly?

[anchapin]

Note that DView opens as expected when the eplusout.sql file is smaller, around 12 MB or less.

[JustinShultz]

I have experienced the same issue.

[nickdiorio]

Thanks for the details, I'll work with @macumber to investigate.

[asparke2]

DView reads the contents of the entire file into memory when the file is opened and must finish this before it can be used. @anchapin I suspect this is taking a long time because your file contains a lot of high-resolution data. Could you please send the eplusout.sql file to andrew.parker@nrel.gov so we can use it for testing?

Unfortunately, loading the file into memory at startup is part of DView's core design, and cannot easily be changed, so we may not be able to speed it up in the short term. This design choice by the original DView developers wasn't an issue because users' files were typically smaller. However, it is easy to generate an eplusout.sql file containing hundreds of series at sub-hourly resolution, especially when looking at EnergyPlus node variables.

Additionally, when loading eplusout.sql files, it is possible to select SI to IP conversion. This conversion happens during the file reading process, and adds even more time.

For the benefit of previous users of OpenStudio ResultsViewer: that program waited to load each series into memory when it was selected for graphing, which is why the file loaded instantly, but each graph took a few seconds to generate.

[anchapin]

My eplusout.sql file is attached to this post. Let me know if you need anything else.
eplusout.zip

I am wondering. Is it possible to only include the Run Period data in the eplusout.sql file? For this project, I have three design days which I don't need to be included in the eplusout.sql file. Note that this run period was only for Jan 1 to Jan 9, when I ran a full year (only one timestep per hour) the eplusout.sql file becomes 862 MB.

[asparke2]

Thanks! Yes, it is possible to only include Run Period data: you can turn this switch to "Off" in the OpenStudio Application.

[shorowit]

One option could be having Dview initially look at which variables are in the sql file, and then ask the user which variables they want to view (dialog box with checkboxes?) so they can downselect from the full set.

[nickdiorio]

@shorowit , yes that sounds like a reasonable option. Another thought is we could load a certain amount of the variables into memory, and if the user selects a variable that is not in memory, we could load it at that point. This would result in some slight delays when plotting those variables, but could result in better overall user experience.

[anchapin]

@asparke2 thanks for showing me that! @shorowit and @nickdiorio, either one of those options sound good to me. I think I would prefer being able to check off which variables to include since I normally only care about 2 or 3 variables each time I use DView.

[luisclv]

It would definitely help to follow the approach of just loading certain variables to memory and then loading the ones that the user selects. Sometimes I do need to see a lot of variables, but not at the same time. I would prefer to wait between changes that not being able to work on the set for 30 minutes.

[anchapin]

An open source alternative to DView, Timestep, was created that specifically addresses this issue according to this comment in unmethours.com. Maybe the source code from Timestep could be used to help fix this bug in DView?

[Ski90Moo]

Is there anybody working on this? I have a ~700mb sql file that takes about 5 hours to load. 700mb is not small, but this seems extremely slow for what it is.