Venom v1.2.0

Major update

Executors

• New mongo executor : https://github.com/ovh/venom/tree/master/executors/mongo
• Executors dbfixtures, exec, http, kafka, web updated.
• Manage secret variables: https://github.com/ovh/venom?tab=readme-ov-file#secrets-variables

Case sensitive

venom 1.2.x

VENOM_PRESERVE_CASE="AUTO" is equals to VENOM_PRESERVE_CASE="ON"

cf. #570

Variable usage

It's no more possible to reuse result.systemout from a previous testcase without exporting vars.

Example

before:

name: A testsuite

testcases:
- name: testA
  steps:
  - type: exec
    script: echo 'foo'
    assertions:
    - result.systemout ShouldEqual foo

- name: testB
  steps:
  - type: exec
    script: echo '__{{.testA.result.systemout}}__'
    assertions:
    - result.systemout ShouldEqual __foo__

after:

name: A testsuite

testcases:
- name: testA
  steps:
  - type: exec
    script: echo 'foo'
    assertions:
    - result.systemout ShouldEqual foo
    vars:
      systemout:
        from: result.systemout

- name: testB
  steps:
  - type: exec
    script: echo '__{{.testA.systemout}}__'
    assertions:
    - result.systemout ShouldEqual __foo__

What's Changed

• doc: add an example for a post using a raw body by @moutyque in #677
• doc: LICENSES by @yesnault in #716
• doc: remove trailing slash on README example by @floriancazals in #693
• doc: Update examples for queries length assert #763 by @miklosbagi in #765
• doc: update README.md with 1.1.0 release version of Venom by @nicolaslacroux in #679
• feat: add a 'description' property by @cdevienne in #770
• feat: add an assertion to check if every element of an array are equal to a value associated to a field by @floriancazals in #756
• feat: add natural language datetime assertion by @rhzs in #711
• feat: add ShouldJSONContain and ShouldJSONContainWithKey and negations (#746) by @floriancazals in #747
• feat: add ShouldJSONEqual assertion by @philippgille in #676
• feat: adding secrets field and refactoring the logging process. by @fokion in #667
• feat: adding test in the root package in order to test the run command by @fokion in #733
• feat: improve junit output by @ivan-velasco in #772
• feat: improve verbose flag behavior by @yesnault in #675
• feat: removed result from testcase variables and removed errors warning and info from non verbose runs by @kilianpaquier in #617
• feat: removed result from testcase variables and removed errors warning and info from non verbose runs by @kilianpaquier in #670
• feat: ShouldMatchRegex by @fokion in #674
• feat: show latest assertions errors in case of retry + added number of retries when success by @kilianpaquier in #616
• feat: VENOM_PRESERVE_CASE default ON by @yesnault in #570
• feat: xmlreport test id by @ivan-velasco in #688
• feat(executor/dbfixtures): add support for sqlite3 to dbfixtures executor by @etandel in #613
• feat(executor/exec): 'command' property by @yesnault in #782
• feat(executor/exec): add a 'stdin' attribute by @cdevienne in #767
• feat(executor/http): allow specifying query parameters by @guoweis-work in #629
• feat(executor/http): feature/xml systemout by @ivan-velasco in #720
• feat(executor/kafka) Added an option to allow self-signed certs by @Armadindon in #648
• feat(executor/kafka): add kafka headers for producer by @rhzs in #710
• feat(executor/mongo): add "loadFixtures" action by @Thiht in #653
• feat(executor/mongo): add mongo executor by @Thiht in #601
• fix: avoid nil pointer dereference in ShouldContain assertions by @iamleot in #615
• fix: env variables with equal sign by @bramca in #721
• fix: godump formatter by @yesnault in #709
• fix: html output color status by @yesnault in #758
• fix: integration tests by @yesnault in #749
• fix: minor improvements by @fsamin in #744
• fix: output dump file by @yesnault in #729
• fix: panic on failure by @yesnault in #731
• fix: parse yaml files in the readfile by @fokion in #689
• fix: prevent err EOF if .venomrc is all commented or empty by @yesnault in #678
• fix: replace backslashes with forward slashes for Windows by @ivan-velasco in #753
• fix: revert #617 by @yesnault in #636
• fix: secrets in dump.json and venom.log files by @yesnault in #715
• fix: string interpolation by @fokion in #656
• fix: TestSuiteXML is missing the Time property by @ivan-velasco in #627
• fix(executor/exec): panic on cmd.Start error by @fehrnah in #623
• fix(executor/http): preserve BodyFile absolute path by @fsamin in #635
• fix(testsuite): when "stop_on_failure" is enabled the skip test cases did not get counted and reported by @ivan-velasco in #771
• refactor(executor): rewrite whole imap executor to add new commands by @tomadele in #611
• refactor(executor/web): Refactoring by @kevinramage in #551
• chore(deps): bump go.mod by @yesnault in #730
• chore(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0 by @dependabot in #761
• chore(deps): bump google.golang.org/grpc from 1.53.0 to 1.56.3 by @dependabot in #739
• chore(deps): bump google.golang.org/grpc from 1.53.0 to 1.56.3 in /tests/grpc by @dependabot in #738
• chore(deps): bump google.golang.org/protobuf from 1.29.0 to 1.29.1 by @dependabot in #647
• chore(deps): bump google.golang.org/protobuf from 1.29.0 to 1.29.1 in /tests/grpc by @dependabot in #646
• chore(deps): bump libs by @yesnault in #644
• chore(deps): bump some libs by @yesnault in #781

New Contributors

• @etandel made their first contribution in #613
• @tomadele made their first contribution in #611
• @fehrnah made their first contribution in #623
• @iamleot made their first contribution in #615
• @ivan-velasco made their first contribution in #627
• @kilianpaquier made their first contribution in #617
• @dependabot made their first contribution in #647
• @Armadindon made their first contribution in #648
• @fokion made their first contribution in #656
• @moutyque made their first contribution in #677
• @nicolaslacroux made their first contribution in #679
• @floriancazals made their first contribution in #693
• @rhzs made their first contribution in #710
• @bramca made their first contribution in #721
• @miklosbagi made their first contribution in #765
• @cdevienne made their first contribution in #767

Full Changelog: v1.1.0...v1.2.0

v1.2.0-beta.4

What's Changed

• feat: new assertion keywords: ShouldJSONContain and ShouldJSONContainWithKey and negations (#746) by @floriancazals in #747
• feat: add natural language datetime assertion by @rhzs in #711
• feat(executor/http): feature/xml systemout by @ivan-velasco in #720
• doc: LICENSES by @yesnault in #716
• fix: secrets in dump.json and venom.log files by @yesnault in #715
• fix(venom): env variables with equal sign by @bramca in #721
• fix: godump formatter by @yesnault in #709
• fix(venom): output dump file by @yesnault in #729
• fix(venom): panic on failure by @yesnault in #731
• fix: it by @yesnault in #749
• fix: minor improvments by @fsamin in #744
• chore(deps): bump google.golang.org/grpc from 1.53.0 to 1.56.3 by @dependabot in #739
• chore(deps): bump google.golang.org/grpc from 1.53.0 to 1.56.3 in /tests/grpc by @dependabot in #738
• chore: bump go.mod by @yesnault in #730
• test: Adding test in the root package in order to test the run command by @fokion in #733

New Contributors

• @bramca made their first contribution in #721

Full Changelog: v1.2.0-beta.3...v1.2.0-beta.4

v1.2.0-beta.3

What's Changed

• Add an example for a post using a raw body by @moutyque in #677
• fix: prevent err EOF if .venomrc is all commented or empty by @yesnault in #678
• docs: update README.md with 1.1.0 release version of Venom by @nicolaslacroux in #679
• fix: Remove trailing slash on README example by @floriancazals in #693
• Add ShouldJSONEqual assertion by @philippgille in #676
• Adding secrets field and refactoring the logging process. by @fokion in #667
• Feature/xmlreport test id by @ivan-velasco in #688
• parse yaml files in the readfile by @fokion in #689
• add kafka headers for producer by @rhzs in #710

New Contributors

• @moutyque made their first contribution in #677
• @nicolaslacroux made their first contribution in #679
• @floriancazals made their first contribution in #693
• @rhzs made their first contribution in #710

Full Changelog: v1.2.0-beta.2...v1.2.0-beta.3

v1.2.0-beta.2

Major update

It's no more possible to reuse result.systemout from a previous testcase without exporting vars.

Example

before:

name: A testsuite

testcases:
- name: testA
  steps:
  - type: exec
    script: echo 'foo'
    assertions:
    - result.systemout ShouldEqual foo

- name: testB
  steps:
  - type: exec
    script: echo '__{{.testA.result.systemout}}__'
    assertions:
    - result.systemout ShouldEqual __foo__

after:

name: A testsuite

testcases:
- name: testA
  steps:
  - type: exec
    script: echo 'foo'
    assertions:
    - result.systemout ShouldEqual foo
    vars:
      systemout:
        from: result.systemout

- name: testB
  steps:
  - type: exec
    script: echo '__{{.testA.systemout}}__'
    assertions:
    - result.systemout ShouldEqual __foo__

What's Changed

• feat: removed result from testcase variables and removed errors warning and info from non verbose runs by @kilianpaquier in #670
• working on ShouldMatchRegex by @fokion in #674
• venom: improve verbose flag behavior by @yesnault in #675

Full Changelog: v1.2.0-beta.1...v1.2.0-beta.2

v1.2.0-beta.1

What's Changed

• feat(executor/dbfixtures): add support for sqlite3 by @etandel in #613
• feat(executor/mongo): mongo executor by @Thiht in #601
• feat(executor/mongo): add "loadFixtures" action by @Thiht in #653
• feat: VENOM_PRESERVE_CASE default ON by @yesnault in #570
• refactor(executor/imap): rewrite whole imap executor to add new commands by @tomadele in #611
• fix(executor/exec): panic on cmd.Start error by @fehrnah in #623
• fix: avoid nil pointer dereference in ShouldContain assertions by @iamleot in #615
• fix: TestSuiteXML is missing the Time property by @ivan-velasco in #627
• feat: removed result from testcase variables and removed errors warning and info from non verbose runs by @kilianpaquier in #617
• fix: revert #617 by @yesnault in #636
• chore(deps): bump libs by @yesnault in #644
• fix(http): preserve BodyFile absolute path by @fsamin in #635
• chore(deps): bump google.golang.org/protobuf from 1.29.0 to 1.29.1 by @dependabot in #647
• chore(deps): bump google.golang.org/protobuf from 1.29.0 to 1.29.1 in /tests/grpc by @dependabot in #646
• feat:(executor/kafka) Added an option to allow self-signed certs by @Armadindon in #648
• feat: show latest assertions errors in case of retry + added number of retries when success by @kilianpaquier in #616
• string interpolation by @fokion in #656
• refactor(executor/web) - Refactoring by @kevinramage in #551
• feat(executor/http): feat: allow specifying query parameters by @guoweis-outreach in #629

New Contributors

• @etandel made their first contribution in #613
• @tomadele made their first contribution in #611
• @fehrnah made their first contribution in #623
• @iamleot made their first contribution in #615
• @ivan-velasco made their first contribution in #627
• @kilianpaquier made their first contribution in #617
• @dependabot made their first contribution in #647
• @Armadindon made their first contribution in #648
• @fokion made their first contribution in #656

Full Changelog: v1.1.0...v1.2.0-beta.1

v1.1.0

Venom v1.1.0

Venom v1.1.0 is here!

Documentation v1.1.0 : https://github.com/ovh/venom/blob/v1.1.0/README.md

Notice that the next version 1.2.0 will preserve case:

venom 1.1.x:

VENOM_PRESERVE_CASE="AUTO" is equals to VENOM_PRESERVE_CASE="OFF"

venom 1.2.x

VENOM_PRESERVE_CASE="AUTO" is equals to VENOM_PRESERVE_CASE="ON"

cf. #570

What's Changed

• feat: Must assertions support by @lowlighter in #463
• feat: skip support at step level by @lowlighter in #592
• feat: add *.yaml extension in venom run usage message by @scraly in #496
• feat: add a registry variable to source executors from by @samj1912 in #571
• feat: add default value for var by @guoweis-outreach in #549
• feat: add feature to produce messages using latest schema available version by @sixstone-qq in #517
• feat: add retry_if support on steps by @lowlighter in #490
• feat: add ShouldTimeEqual assertion by @marcaudefroy in #506
• feat: add support for sqlite in sql executor by @maciekzieba in #598
• feat: allow +json in contentType by @guoweis-outreach in #528
• feat: cache executor files by @philippgille in #553
• feat: detailed steps output by @lowlighter in #487
• feat: handle param like charset by @guoweis-outreach in #550
• feat: improve output with range by @yesnault in #588
• feat: official docker image by @fsamin in #567
• feat: output a test_result files per testsuite by @yesnault in #560
• feat: output html by @yesnault in #572
• feat: preserve case by @yesnault in #569
• feat: run testcase with assertion only by @fsamin in #468
• feat: Upgrade Kakfa client version to 2.6.0 by @sarunask in #456
• feat: Venom: Support for .yml & .yaml extensions by @scraly in #495
• feat(assertions): add logical operators by @lowlighter in #455
• feat(executor/http): various improvements by @fsamin in #481
• feat(grpc) tls by @fabriziomoscon in #462
• feat(ssh_executor): Sudo option by @louisquentinjoucla in #488
• feat(tests): add range supports by @lowlighter in #453
• chore: allow darwin arm by @yesnault in #555
• chore: bump interpolate lib by @yesnault in #505
• chore: enable git pod by @fsamin in #559
• chore: make clean + golang 1.17 for integration tests by @fsamin in #480
• chore: Makefile - fix cross compilation by @yesnault in #509
• chore: use go install by @yesnault in #554
• doc: testsuite sort on venom run by @yesnault in #474
• doc: update readme with new usage messages by @scraly in #497
• doc: variables helpers examples by @yesnault in #568
• doc(imap): port example by @yesnault in #596
• doc(contrib): review of readfile/README.md by @s47roy in #538
• doc(contrib): typo and review of exec/README.md by @s47roy in #540
• doc(contrib): typo and review of executors/README.md by @s47roy in #541
• doc(contrib): typo and review of README.md by @s47roy in #544
• doc(contrib): typo and review of web/README.md by @s47roy in #543
• doc(contrib): typo and review of web/web.go by @s47roy in #542
• fix absolute file path for gRPC TLS certs by @sarunask in #513
• fix invalid session #518 by @kevinramage in #519
• fix json quote by @fsamin in #586
• fix small typo by @sapk in #516
• fix waitFor and messagesLimit behaviour in Kafka consumer by @sixstone-qq in #584
• fix: [redis]: Use Readline to read large value by @marcaudefroy in #471
• fix: assertion with json number by @fsamin in #546
• fix: compute stats ok/ok/skipped on json format by @fsamin in #477
• fix: disable colors in venom.log file by @fsamin in #562
• fix: don't write invalid characters in report files by @fsamin in #482
• fix: Ensure the output file has '.xml' extension by @clement-sciascia in #466
• fix: handle executor setup errors by @fsamin in #561
• fix: handle json number is assertions (#460) by @yesnault in #464
• fix: handle object and array are part of user executor result by @fsamin in #566
• fix: Improve linter configuration by @fabriziomoscon in #461
• fix: Improve UX & DX in Venom CLI by @scraly in #491
• fix: Improvments in README by @scraly in #489
• fix: interpolate arg var only once by @yesnault in #563
• fix: interpolation lib bad µµµ replacement by @fsamin in #479
• fix: line numbers being duplicated by @seanlane in #504
• fix: link in Readme by @komalsukhani in #600
• fix: multilines in args user executor by @yesnault in #565
• fix: printing with Verbose by @yesnault in #587
• fix: step variables inheritance leak by @fsamin in #545
• fix: typos by @fsamin in #593
• fix(executor/exec): manage \n in xml output by @yesnault in #576
• fix(executor/http): isBodyJSONSupported by @fsamin in #483
• fix(executor/http): preserveBodyFile interpolation by @yesnault in #575
• refactor: --html-report instead of format=html by @yesnault in #580

New Contributors

• @clement-sciascia made their first contribution in #466
• @scraly made their first contribution in #489
• @sarunask made their first contribution in #456
• @seanlane made their first contribution in #504
• @louisquentinjoucla made their first contribution in #488
• @sapk made their first contribution in #516
• @guoweis-outreach made their first contribution in #528
• @s47roy made their first contribution in #538
• @samj1912 made their first contribution in #571
• @komalsukhani made their first contribution in #600
• @maciekzieba made their first contribution in #598

Full Changelog: v1.0.0...v1.1.0

v1.1.0-rc.1

What's Changed

• feat: skip support at step level by @lowlighter in #592
• fix: typos by @fsamin in #593
• doc(imap): port example by @yesnault in #596

Full Changelog: v1.1.0-beta.6...v1.1.0-rc.1

v1.1.0-beta.6

What's Changed

• refactor: --html-report instead of format=html by @yesnault in #580
• Fix waitFor and messagesLimit behaviour in Kafka consumer by @sixstone-qq in #584
• Fix json quote by @fsamin in #586
• fix: printing with Verbose by @yesnault in #587
• feat: improve output with range by @yesnault in #588

Full Changelog: v1.1.0-beta.5...v1.1.0-beta.6

v1.1.0-beta.5

What's Changed

• doc: variables helpers examples by @yesnault in #568
• feat: preserve case by @yesnault in #569
• Add a registry variable to source executors from by @samj1912 in #571
• feat: output html by @yesnault in #572
• fix(executor/http): preserveBodyFile interpolation by @yesnault in #575
• fix(executor/exec): manage \n in xml output by @yesnault in #576

New Contributors

• @samj1912 made their first contribution in #571

Full Changelog: v1.1.0-beta.4...v1.1.0-beta.5

v1.1.0-beta.4

What's Changed

• chore: allow darwin arm by @yesnault in #555
• feat: detailed steps output by @lowlighter in #487
• chore: enable git pod by @fsamin in #559
• feat: output a test_result files per testsuite by @yesnault in #560
• fix: handle executor setup errors by @fsamin in #561
• fix: disable colors in venom.log file by @fsamin in #562
• fix: interpolate arg var only once by @yesnault in #563
• fix: handle object and array are part of user executor result by @fsamin in #566
• fix: multilines in args user executor by @yesnault in #565
• feat: official docker image by @fsamin in #567

Full Changelog: v1.1.0-beta.3...v1.1.0-beta.4