Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regression checklist template #536

Closed
25 tasks
tyuldashev opened this issue Nov 14, 2022 · 1 comment · Fixed by #589
Closed
25 tasks

Regression checklist template #536

tyuldashev opened this issue Nov 14, 2022 · 1 comment · Fixed by #589
Assignees
@tyuldashev
Labels
qa Issue is related to QA process

Comments

@tyuldashev
Copy link
Collaborator

tyuldashev commented Nov 14, 2022 •
edited
Loading

Get build from this run - https://github.com/UnitTestBot/UTBotCpp/actions/workflows/publish-utbot.yml

Setup dedicated server according to following manual - https://github.com/UnitTestBot/UTBotCpp/wiki/install-server
Setup local server according to following manual - https://github.com/UnitTestBot/UTBotCpp/wiki/docker-free-wsl2

Delete project folders with sample project on remote and local server if they are left from previous test runs.

VSCode:

  • Install plugin (Ctrl-Shift-P - Extensions: Install plugin from VSIX)

  • Verify version on plugin page (Ctrl-Shift-X search for @installed unittestbot)

  • Open folder with unpacked attached sample project - UnitTestBot Wizard should be started

  • SFTP and Sarif Viewer plugins are marked as installed on first page of the Wizard

  • On Server configuration check that both local and server connection could be established.

  • When cannot connect proper error should be shown

  • Connect to remote server and complete the wizard - build folder created, project configured

  • Open lib\calc.c file, generate tests for div(int a, int b) function - tests are generated, Sarif report is opened

  • Run only one test - regression tests pass, error tests fails, coverage shown in calc.c file.

  • Run all generated tests in test file - regression tests pass, error tests fails, coverage shown in calc.c file.

  • Generate and run tests for items listed below. Before the test remove tests folder. After test verify proper number of tests files appeared and run one of them.

    • File lib\str_utils.c
    • Folder src
    • Project

  • Verbose

    • On
    • Off

  • Log windows (writing to log could be unstable - see [BUG] No output generated for 'UTBot Server Log' #430)

    • UTBot: Client Log
    • UTBot: Server Log
    • UTBot: Test console

  • Open UTBot Explorer (at left pane)

  • CLion plugin - do simple test - install plugin - configure to work with local server generate and run some test. Additionally verifying that coverage is generated.
@tyuldashev tyuldashev added the qa Issue is related to QA process label Nov 14, 2022
@tyuldashev tyuldashev self-assigned this Nov 14, 2022
@tyuldashev tyuldashev moved this to Todo in UTBot C/C++ Nov 14, 2022
@ladisgin ladisgin linked a pull request Feb 28, 2023 that will close this issue
Add regression checklist template #589
Merged
@ladisgin
Copy link
Member

#589

@github-project-automation github-project-automation bot moved this from Todo to Done in UTBot C/C++ Feb 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tyuldashev tyuldashev

Labels
qa Issue is related to QA process
Projects
UTBot C/C++
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add regression checklist template UnitTestBot/UTBotCpp
2 participants
@tyuldashev @ladisgin