Data driven execution should be represented in context

[jensakejohansson]

I'm running a specification using data driven execution. I will not go into the exact details of what I'm trying to achieve, but I do believe that the table data that that gets "injected" into each execution of a scenario should be available in it's context in the execution hooks.

Basically in the [BeforeSpec] (I'm using dotnet):

[BeforeSpec]
public void BeforeSpec(ExecutionContext context)

I'd like to have a reference to the current data from the "data driven"-table (essentially the current row of the table) and also be able to dynamically modify it (add colums, alter column data) and any changes should of course be propagated through out the scenario.

This should give the user much more freedom to extend the data driven possibilities by inject "dynamic" data, inject test data from other sources on-the-fly and so on.

[sriv]

I imagine that the context can be enriched with the run being data driven, and more information about current row and the resolved data row etc.

This should give the user much more freedom to extend the data driven possibilities by inject "dynamic" data, inject test data from other sources on-the-fly and so on.

This goes against the responsibility designed, i.e mutating the context bit, not empowering the user bit. The spec files and supporting data files are the source of record for the data driven execution.

I believe gauge could do better to support more data sources and allowing users to plug in richer data source. We discussed this, but at that time it came out that this will cause other aspects (ex. gauge's parsing) to break, so we parked it.

[jensakejohansson]

Agreed that changing the data in hooks would likely violate the intended design in terms of responsibility and design patterns.

Hm, it would not be reasonable to allow a "data-setup-method"? Instead of a table/text-file at the top of the spec, as of now, you could also have "data-setup-step" in which the user could generate the test-data with the requirement that the method must return a Gauge-table? The table would then been used in the same way as today. Then it would be possible to programtically generate test-data in that step definition, and after that point no mutating of the data is possible - but still it should be readable through context. I believe context should provide as much information as possible about what's going on.

I guess what I'm asking for is basically something along the line of what it is provided by for example TestCaseSource in NUnit:
https://docs.nunit.org/articles/nunit/writing-tests/attributes/testcasesource.html

[sriv]

yeah, that is also something I had thought of. Essentially similar to nunit, the idea is to have a [TestDataSource] attribute, that would decorate a method which returns a DataTable.

The challenge here is internal to gauge's architecture, but the gist is:

• gauge requires the test data to perform it's validation
• gauge's validation has to run without bringing up the runner to perform validations that do not require a runner
• the TestDataSource approach essentially will require gauge to bring up runner before the validation

Not saying this isn't possible or should/can not be done. Just that it requires change at different level, which will impact all runners, so this is a not-so-insignificant change. Bandwidth is a challenge for me personally, but I am more than happy to help in case anyone is willing to drive this.

[jensakejohansson]

Hi, @sriv We're thinking about looking into parts of this issue ourselves when we have time and do a PR.

1. Enriching the context with information about the current test data row. So you can access all test data everywhere in the specification (as long as you have a reference to a context object).

2. Somehow enriching reporting with test data information. The xml-reporter for example shows very primitive information about data driven tests. If I remember correctly it just numbers the executions with 1,2,3,... etc.

Btw, it could also be that some of the PRs we suggest should include some small updates in the web-based documentation? What's the procedure to do that?

Best regards,

[zabil]

small updates in the web-based documentation? What's the procedure to do that?

Info about the how to update documentation site is available at
https://github.com/getgauge/docs.gauge.org