-
Notifications
You must be signed in to change notification settings - Fork 263
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add sim_opt 'ghdl.elab_e' to run 'ghdl -e' instead of 'ghdl --elab-run --no-run' #467
Conversation
The point of the acceptance test is to define the behaviour of VUnit as a black box. It should work for any simulator we support. Unfortunately we are only able to run the test on Travis CI for GHDL. The elab fail test will test that an error at the elaboration stage is caught when running with the |
I think that the issue is whether those tests were/are meaningful in the context of 'elaboration only'. The fact that I believe that the acceptance tests which were expected to fail before but do not fail now, have never been testing elaboration options, but runtime arguments. They do not fail now because those args are not passed anymore, as they do not belong. Therefore, I suggest to move some of the tests in
|
The reason the elab_fail test bench fails is because it has a generic for which no value is specified. Currently VUnit offers two stages of running a test bench, elaborate only or full test run. What would be the stage in-between expressed in the terms of the VHDL-standard? Run only the first delta cycle? Is this what -r --no-run does? Also if we add a new stage it needs to be implemented for all simulators so it becomes more work. There are two ways to solve this on the user-side:
Maybe 1. is the best method. |
I did not know that other simulators do check generics at elaboration. I wonder why GHDL is not consistent. Is there any requirement for this PR to be accepted other than "GHDL should evaluate generics at
When the top-level of the binary is GHDL, existing two options behave as expected: elaborate generates the binary and the default executes it. The fact that However, when the simulation is wrapped, there is no option in VUnit to just check if the design can be compiled and build the binary. I believe that, in terms of the VHDL standard, this stage is the output of the elaboration. Ideally, I do not want to run until the first delta cycle, I don't want to execute the binary at all.
If there is no requirement other than evaluating generics during elaboration, this PR will eventually be valid, so ignore the following paragraphs. Otherwise, if I don't think that the new option needs to be added for all simulators. This is only required for the simulators that allow to build an executable binary which includes some VHDL sources/objects and some C sources/objects. What do you think about On the one hand, this is only supported by GHDL for now, so the implementation in other simulators would be an error telling that it is a not supported arg. On the other hand, although some other simulators supported by VUnit allow to combine VHDL and C sources (e.g. https://gitlab.com/reds-public/fss), I don't know if any of them allows to generate an independent executable binary. If it is not the case, this would always be a GHDL-only flag.
I'd rather call |
Elaboration in my understanding means executing all code that results in a compile time constants such as the choice in an if-generate statement, bit width of a port or signal etc. This is what most simulators do when loading the design. This is different from executing the first delta cycles of all processes. It is also one of the earliest stages of a synthesis tool. A synthesis tool will never execute a process but it will execute all code that results in constant values such as bit widths and initial values. This is what I would expect should happen when i run "ghdl -e" if e means elaborate. It might be that today we are doing to much in the VUnit elaboration stage for GHDL if also the first delta cycle is executed (equivalent with running "run 0 ns") in other simulators. However I would rather err on this side than having less checks for --elaborate. Thus I would prefer to keep elaborate the way it is now. If in the future GHDL adds more elaboration into its "-e" flag maybe we can switch to use it. My thinking is still if this is not best fixed on outside of VUnit by simply disabling the post simulation check in the user binary if '--no-run' was in ARGV. Conceptually it is a simple operation and it does not require VUnit to add any GHDL specific --build flag or reduce the amount of verification provided by --elaborate. Conceptually I am not totally against adding a --build flag I am just thinking if it is pre-mature to do that at this point. VUnit should not stand in the way for the user to do anything he wants and it does not in this case, it does not that VUnit shall have every possible wish built into it that can be solved on the user side. |
After a more careful reading of the docs, it seems that
Therefore, it seems that there is no alternative other than running
Yes, I was considering to keep this issue open just in case. However, enough features are not likely to be added to the elaboration command in the near future. Therefore, we can close this either now or after we are done with the discussion.
It is possible to do it in several ways that do no require modification of the application (executing
I'll try to check if I can retrieve the simulation command generated in GHDLInterface from the |
I think I can achieve this by adding a However, I need to retrieve the logs, in order to manipulate them and generate the build commands in the |
If you want to know the output path for a test you need to use the Regarding the issue at hand, if you really want |
3dea5ac
to
9cdb9c9
Compare
It is possible to read
It would be useful to retrieve the simulation command with all the arguments, just as it would be executed by VUnit. However, the use case for that is far beyond the scope of VUnit. So, it is ok to just be able to build the binary without explicitly seeing the command. I implemented the approach you proposed (a |
The file was intended for human use. The format is
Currently the commands are logged if you use --log-level=DEBUG for human use. If an external tool needs this information we need to add it to
I think this should be a unit test of ghdl_interface.py. It is to complex to have an acceptance test for it, the ROI is not worth it. |
Reviewing your change it seems I was misunderstood. I meant the sim_option would change the behaviour of |
IMHO, there should be a simple and realiable programatic procedure to retrieve the output/log of a test. It don't really mind if I need to set some stdout in the python script, or get it as a return field from About adding it to
The purpose of this external tool is to provide the objects produced by
I will add it there.
I will adapt it. |
I modified the PR to let I also added a test, |
You are mocking the wrong function. The failure occurs at: vunit/ghdl_interface.py:253: in simulate
proc = Process(cmd)
vunit/ostools.py:131: in __init__
preexec_fn=os.setpgrp) # pylint: disable=no-member An alternative to mocking is separating the code that generates the command from the code that executes the command and only test the code that generates the command. |
I did that now. |
The AppVeyor (Windows tests) fail because you use a hardcoded POSIX path with forward slash / in your unit test whereas on windows it will become a backward slash . You should use os.path.join in the unit test to get a platform agnostic path. |
Thanks! It should be fixed now. |
Close #466.
EDIT
Since, VUnit expects GHDL to actually evaluate some flags and generics during elaboration-only, the first commit in this PR breaks
test_artificial_elaborate_only
. So, in the second commit the tests are changed, and all of them are expected to pass now. I don't know if this makes sense. Probably, some of these tests should be removed (config.sim_options
,config.generics
andconfig.vhdl_assert_stop_level
are ignored now). However, I am unsure because they might be used with other simulators.