Assert Error and simulation failed and NVC vs the world

[JimLewis]

Most simulators allow you to set how the simulator reacts to Assert ERROR vs Assert FAILURE.

Currently in NVC we are seeing that an Assert ERROR triggers the simulator to exit with simulation failed status. Other simulators do not do this - at least with the settings we use by default.

The question is, do we tell NVC not to do that, or do we decide that this is a good thing and change the settings for other simulators so they also produce a simulation failure?

Thoughts? @bpadalino @nickg @Paebbels @SkydiverTricky @tgingold

[nickg]

Is it really the only simulator that does this? I'm sure I would have copied that behaviour from somewhere, but it's been a long time since I used any commercial simulator. Personally I find it useful to abort immediately on the first assertion failure without having to add severity failure. It's also possible to override this by passing --exit-severity=failure.

[JimLewis]

@nickg I tested GHDL, NVC, RivieraPRO and Questa. Only NVC. Not saying it is wrong. My usage of assert is a special case. It only impacts 3 tests. They can be skipped. So I am not racing off to change the behavior one way or another. Just wondering what others think.

[SkydiverTricky]

I assume we are talking about exit codes for allowing pass/fail detection in a CI setup? I raised a ticket about this with Aldec several years ago about throwing the exit code for std.env.stop(N) and assert failures and they closed the ticket claiming it was fixed (or added) in ActiveHDL 13.0. Not sure how it would work with assert ERROR though

I also had a discussion with an engineer at Seimens about it - they dont seem to want to do it at all. They were talking about having to call some custom seimens package to do it (that sounds horrible). Either that or didnt understand what/why I was asking for it. It all has to marry with how SV works apparently.

[SkydiverTricky]

I currently have TCL code that interrogates the AlertLogPkg for the Alert Counts and also scrubs the logs for assert failures.