Flow fails loading and recursively removes EGRID file

[bludvigsen]

When attempting to load EGRID file with the GDFILE keyword, Flow fails with the following error message and recursively deletes the EGRID file:

Error:
An error occurred while creating the reservoir properties
Internal error: Can not open EclFile: IVAR_AASEN_2021_REF.EGRID

Error: Unrecoverable errors while loading input: Can not open EclFile: IVAR_AASEN_2021_REF.EGRID

[bska]

If you're running a fairly recent version of the simulator, this may be unforeseen fallout from some work we did earlier this year. If your casename is IVAR_AASEN_2021_REF.DATA–i.e., if the name of the .DATA file matches the name of the .EGRID file in question–then there's a possibility that we're being a little too aggressive in pruning the result file output directory at simulation startup.

What does your GDFILE statement look like? Is it

GDFILE
  'IVAR_AASEN_2021_REF.EGRID' /

or is it more like

GDFILE
  'some/directory/IVAR_AASEN_2021_REF.EGRID' /

?

[blattms]

Well, it means that we can't read that file. Are you sure that the file is there. Please take into account that one Linux file names are case sensitive. Maybe the case is different. Also the path specified needs to be relative to the directory where the *.DATA file resides.

Can you go to the directory where the *.DATA file is and execute ls? You should see the file in the output.

[bludvigsen]

Yes, the GDFILE is:

GDFILE
'IVAR_AASEN_2021_REF.EGRID' /

Yes, the file is there before executing 'flow', and after executing it is deleted in the current directory and directories below. It is not deleted in the directory above. The pruning should just be in the cwd where the results are.

I am converting an Eclipse case to OPM and I have the original model stored in a folder called 'Eclipse' in cwd. The EGRID file in this directory is deleted along with the EGRID file in cwd, which is not good.

I could try './IVAR_AASEN_2021_REF.EGRID' and see if that helps but it reads the other files fine which are defined in the same way. If it does not work I will make GRDECL files with ZCORN and COORD and let OPM build the grid.

[bludvigsen]

Just figured out that it treats the EGRID file in cwd as a result file and deletes it before it tries to read it, does that make sense? Solution could then be to have an include directory, above the cwd.

[bska]

I am converting an Eclipse case to OPM and I have the original model stored in a folder called 'Eclipse' in cwd. The EGRID file in this directory is deleted along with the EGRID file in cwd, which is not good.

Okay, understood. Thanks a lot for the information. Just one more thing if you don't mind. Is your .DATA file also named

IVAR_AASEN_2021_REF.DATA

in the same directory as the existing grid file?

Just figured out that it treats the EGRID file in cwd as a result file and deletes it before it tries to read it, does that make sense?

If your .DATA file is indeed named IVAR_AASEN_2021_REF.DATA and is located in the same directory as the IVAR_AASEN_2021_REF.EGRID file, then yes that is what happens and it's caused by the work I alluded to earlier.

Solution could then be to have an include directory, above the cwd.

You mean something like

GDFILE
  '../egrid/IVAR_AASEN_2021_REF.EGRID' /

? If so, that would definitely do the trick. Another option, if it's possible in your workflow, would be to have the .DATA file not share the same name as the input .EGRID file.

I could try './IVAR_AASEN_2021_REF.EGRID' and see if that helps

That would probably not help, because the simulator will prune any (potential) output file, in the same directory as the .DATA file, with the same case name on simulation startup.

[bludvigsen]

Yes, now it worked with this setup:

GDFILE
'../IVAR_AASEN_2021_REF.EGRID' /

Yes, good idea also with different name.

Thanks for your support!

[atgeirr]

It is good that you have a workaround! However it seems to me we still have to solve the problem of Flow deleting files it should not delete. Any idea @bska or @blattms?

[bska]

However it seems to me we still have to solve the problem of Flow deleting files it should not delete.

Yeah... We can solve this particular instance, and I believe it's the most common one as well, by not deleting CASE.EGRID when running CASE.DATA. In that case we'll effectively end up rewriting CASE.EGRID if CASE.DATA has

-- ...
GRID
-- ...
GRIDFILE
 0 1 /
-- ...

I'm more hesitant about the general problem. Our "delete result files on startup" routine uses a regular expression match to identify potential result files and any files matching those will be pruned very early in the simulation process. This behaviour was introduced in PR #5168.

I suppose we could defer pruning the result files until after we've loaded the input deck and know all INCLUDE, IMPORT, and GDFILE keywords, but then we need to be careful that we don't delete any files currently being written. For what it's worth I would really like to keep the "prune potential result files" behaviour because that helps with/solves another big problem.

[blattms]

Maybe we could add a stamp -- Written by OPM-flow at ... to the beginning of files that we write to? Then we could really judge whether a file was written by flow and only delete that.

On the other hand we can recheck which files caused problems. I doubt that it was the EGRID file that caused problems. Must have been something that we read also from during output.

[bska]

Maybe we could add a stamp -- Written by OPM-flow at ... to the beginning of files that we write to?

That would be easy to do for the .PRT and .DBG files, but a lot harder for the unformatted result files (.EGRID, .INIT and so on). Those have to adhere to a rather strict protocol in order to match expectation of post-processing software such as ResInsight.

On the other hand we can recheck which files caused problems. I doubt that it was the EGRID file that caused problems. Must have been something that we read also from during output.

Would you care to elaborate a little on this, please? The setup in this case is CASE.DATA using

GDFILE
  'CASE.EGRID' /

in its GRID section. In other words, the name of the input EGRID file is the same as the name of the case file itself. Since we prune all (potential) result files–including CASE.EGRID–as part of setting up our logging system, i.e., before we open/read the input file, that file removal also removes the requisite CASE.EGRID input file.

[blattms]

I was not talking about this case. I was talking about why we started removing files. That was because we got:

Error: [opm/io/eclipse/EclUtil.cpp:293] Error reading binary header. Expected 16 bytes of header data, found 4000
ERROR: Uncaught std::exception when running tasklet: [opm/opm-common/src/opm/io/eclipse/EclUtil.cpp:293] Error reading binary header. Expected 16 bytes of header data, found 4000. Trying to continue.

If we know for what files this can happen for, then we could only delete those.

But I guess you were suggesting to not use known output files for input because they will be overwritten. That might cause problems anyway, right? Should we detect this and abort?

[mantegaisa]

Hello.

I joined this conversation because of an error I got when simulating a case. The terminal returned the following message:

Error: [./src/opm/io/eclipse/EclUtil.cpp:293] Error reading binary header. Expected 16 bytes of header data, found -1019319855

What can be the reason for getting this message? There are no output files being used for input.

Thanks in advance.

[bska]

What can be the reason for getting [the "Error reading binary header"] message?

The most common cause of that diagnostic is running a sequential Flow binary in parallel. Are you running simulations in parallel using mpirun -np N with N being an integer strictly greater than one?

[bska]

The most common cause of that diagnostic is running a sequential Flow binary in parallel. Are you running simulations in parallel using mpirun -np N with N being an integer strictly greater than one?

Quick follow-up: Another potential issue is the simulator expecting an unformatted input file, e.g., in IMPORT or GDFILE, and then being presented with a formatted one instead. Please expand a little on what your setup is, how you're running the simulator and any diagnostic messages you might have gotten just prior to getting the error mentioned earlier. Unless the model is confidential, you might also wish to post the .DBG output file here.