Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run test suite on windows during CI #6648

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
83dee1f
Bump the Qt version from 6.4.3 to 6.5.3.
brettle Aug 15, 2024
473ce86
Add changelog entry.
brettle Aug 15, 2024
068d715
Merge branch 'master' into fix-to-use-supported-qt-version
brettle Aug 15, 2024
8293c4c
Get Qt from system on ubuntu 22.04 and via aqtinstall on Ubuntu 20.04.
brettle Aug 17, 2024
bc574a2
Move package installation to linux_compilation_dependencies.sh and li…
brettle Aug 17, 2024
6de0828
Fix package name for Qt WebSockets on Ubuntu 22.04.
brettle Aug 17, 2024
b192745
Temporarily move Mac back to Qt 6.4.3 in hopes of getting the test su…
brettle Aug 17, 2024
ef65c5d
Temporarily allow access for debugging.
brettle Aug 18, 2024
79b6b62
Use aqt for all linux because Ubuntu 22.04 only has Qt 6.2 and we nee…
brettle Aug 18, 2024
bd9f821
Add back libxcb-cursor0.
brettle Aug 18, 2024
19287c6
Add openssl as a runtime dep.
brettle Aug 18, 2024
f7a8341
Copy system openssl3 libs on Ubuntu 22.04 and use built libs on Ubunt…
brettle Aug 18, 2024
67e9f26
Build openssl3 from source on Ubuntu 20.04.
brettle Aug 18, 2024
e672407
Get rid of unneeded and unrecognized pushd in Makefile.linux.
brettle Aug 18, 2024
87fa748
Use aqt to install Qt 6.5.3 for mac.
brettle Aug 19, 2024
8a49ced
Allow login to mac if build fails.
brettle Aug 19, 2024
eac55f7
Make sure the Contents/Frameworks folder is created.
brettle Aug 19, 2024
3d313b6
Remove upterm steps that were added for debugging.
brettle Aug 19, 2024
37c0f00
Make openssl just a compilation dependency for Ubuntu 22.04 because w…
brettle Aug 19, 2024
71d6a43
Try to run the test suite during macos CI.
brettle Aug 19, 2024
f1d55d3
Set WEBOTS_HOME_PATH such that tests can find Makefile.include during…
brettle Aug 19, 2024
f4bbf59
Allow login for debugging.
brettle Aug 19, 2024
a0ab607
Change WEBOTS_HOME to Webots.app and add logic to test_suite.py to fi…
brettle Aug 19, 2024
86873a1
Get qt_warning_filters.conf from correct location on macos.
brettle Aug 19, 2024
9417594
Remove accidentally added checkout step.
brettle Aug 19, 2024
d95696e
Hopefully squash clang-format complaint.
brettle Aug 19, 2024
70f7d7b
Ignore (but warn about) error trying to connect to background Webots …
brettle Aug 20, 2024
be29ef1
Fix formatting.
brettle Aug 20, 2024
9488367
On Mac, timeout test after 60 minutes instead of 10.
brettle Aug 20, 2024
f099a99
Merge branch 'master' into fix-test-suite-not-run-on-macos
brettle Aug 25, 2024
4954b9c
Try single threaded build on macos CI.
brettle Aug 26, 2024
554a9cb
Try using macos-13 runner instead of macos-14. The former has twice a…
brettle Aug 26, 2024
60f7682
Merge branch 'master' into fix-test-suite-not-run-on-macos
brettle Aug 27, 2024
4a1c4da
Install urdfdom during macos CI because it is needed by tests suite.
brettle Aug 27, 2024
5e26dbd
Merge branch 'master' into fix-test-suite-not-run-on-macos
brettle Aug 27, 2024
c0c2150
Assert that move succeeds.
brettle Aug 28, 2024
b51cef0
Print age of ode_tmp.dif so we can see how long we should wait on mac…
brettle Aug 28, 2024
892614a
Allow up to 20 seconds for world reload.
brettle Aug 28, 2024
9d850f8
Try waiting longer before timing out.
brettle Aug 29, 2024
ee341c4
Skip billboard.wbt test during macos CI.
brettle Aug 29, 2024
418c422
Restore multithreaded build.
brettle Aug 29, 2024
553c537
Remove unneeded comment.
brettle Aug 29, 2024
b301158
Remove upterm.
brettle Aug 29, 2024
1468e14
Only increase timeout on macos.
brettle Aug 29, 2024
6fbe675
Simplify some syntax.
brettle Aug 30, 2024
60fdb16
Only run the test suite if the "test suite mac" label is present.
brettle Aug 30, 2024
8d2abb2
Add a test-suite job.
brettle Aug 31, 2024
4e3bf02
Upload artifact after build if test suite requested.
brettle Aug 31, 2024
8fb2814
Add upterm for remote debugging.
brettle Aug 31, 2024
58c0145
Remove upterm (not supported on Windows) and use correct name for set…
brettle Aug 31, 2024
0f1b2df
Use exact name for installer.
brettle Aug 31, 2024
081d8de
Use relative path.
brettle Aug 31, 2024
2976d09
Setup msys2, don't delete articacts, and ls the artifact director.
brettle Aug 31, 2024
c28d8f7
Merge remote-tracking branch 'origin/master' into fix-test-suite-not-…
brettle Aug 31, 2024
82fd6ed
Use wildcard name again.
brettle Aug 31, 2024
6adb9da
Disable build temporarily.
brettle Aug 31, 2024
db0052d
Disable needs build temporarily.
brettle Aug 31, 2024
0b2cc3f
Update test_suite_windows.yml
brettle Aug 31, 2024
f9c74e8
Install deps.
brettle Aug 31, 2024
db7f563
Install VS10 and OpenJdk so test-suite can run.
brettle Aug 31, 2024
980e7b0
Set WEBOTS_HOME.
brettle Sep 1, 2024
6a501fe
Add WEBOTS_HOME to PATH before running tests.
brettle Sep 1, 2024
616e97a
Fix PATH.
brettle Sep 1, 2024
2488237
Print sysinfo output when error occurs.
brettle Sep 2, 2024
f2d43b5
Print debug info when getting sysinfo.
brettle Sep 3, 2024
ded17b5
More debugging output.
brettle Sep 3, 2024
44ee4a1
Install software renderer for headless CI.
brettle Sep 3, 2024
81eac77
Launch tmate session for debugging.
brettle Sep 3, 2024
71ce28f
Fix syntax.
brettle Sep 3, 2024
636a5fe
Replace debugs with asserts when gathering sysinfo.
brettle Sep 3, 2024
921c14c
Setup tmate sessoin if the test suite fails.
brettle Sep 3, 2024
15b529c
Use more consistent syntax in if clause.
brettle Sep 3, 2024
bee3117
Read MyBot.txt in text instead of binary mode to compensate for line …
brettle Sep 3, 2024
cd5e89c
Treat hdr files as binary so they aren't corrupted by git when checki…
brettle Sep 4, 2024
d3543cb
Create opengl context even during non-debug builds.
brettle Sep 6, 2024
af436e3
Merge remote-tracking branch 'origin/master' into fix-test-suite-not-…
brettle Sep 6, 2024
9ccc2e6
Use download artifact v4.
brettle Sep 6, 2024
d8f6ff9
Override GL version to be 3.3 and force software rendering.
brettle Sep 6, 2024
d73a1d1
Skip a couple tests that don't work on windows CI.
brettle Sep 6, 2024
2044482
Make test_suite.py accept world files under windows.
brettle Sep 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 64 additions & 2 deletions .github/workflows/test_suite_windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,13 +88,75 @@ jobs:
python -m pip install requests PyGithub
scripts/packaging/publish_release.py --key=${{ secrets.GITHUB_TOKEN }} --repo=${{ github.repository }} --branch=${{ github.ref }} --commit=$(git log -1 --format='%H') --tag=${{ github.ref }}
- uses: actions/upload-artifact@v4
if: ${{ contains(github.event.pull_request.labels.*.name, 'test distribution') }}
if: ${{ contains(github.event.pull_request.labels.*.name, 'test suite') || contains(github.event.pull_request.labels.*.name, 'test distribution') }}
with:
name: build-${{ matrix.os }}
path: |
distribution/*.exe
test-suite:
needs: build
if: ${{ contains(github.event.pull_request.labels.*.name, 'test suite') }}
strategy:
matrix:
os: [windows-2019]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- name: Download Artifacts
uses: actions/download-artifact@v4
with:
name: build-${{ matrix.os }}
path: artifact
- uses: msys2/setup-msys2@v2
with:
update: false
- name: Install Visual Studio 10 and OpenJDK 18
shell: powershell
run: |
choco install -y openjdk --version=18.0.2
choco install -y visualcpp-build-tools
- name: Install Webots Compilation Dependencies
run: |
export PYTHON_INSTALLATION_FOLDER=/C/hostedtoolcache/windows/Python
export PYTHON_HOME=$PYTHON_INSTALLATION_FOLDER/3.11.`ls $PYTHON_INSTALLATION_FOLDER | grep '^3\.11\.[0-9]\+$' | cut -c6- | sort -n | tail -n1`/x64
echo 'export JAVA_HOME=/C/Program\ Files/OpenJDK/`ls /C/Program\ Files/OpenJDK`' >> ~/.bash_profile
echo 'export PYTHON_HOME='$PYTHON_HOME >> ~/.bash_profile
echo 'export VISUAL_STUDIO_PATH="/C/Program Files (x86)/Microsoft Visual Studio/2017"' >> ~/.bash_profile
echo 'export INNO_SETUP_HOME="/C/Program Files (x86)/Inno Setup 6"' >> ~/.bash_profile
echo 'export PATH=$PYTHON_HOME:$PYTHON_HOME/Scripts:$GITHUB_WORKSPACE/msys64/mingw64/bin:$GITHUB_WORKSPACE/bin/node:/mingw64/bin:/usr/bin:$JAVA_HOME/bin:$PATH' >> ~/.bash_profile
export WEBOTS_HOME=$GITHUB_WORKSPACE
./scripts/install/msys64_installer.sh --all
- name: Extract Webots
run: |
./artifact/webots-*.exe //SUPPRESSMSGBOXES //VERYSILENT //NOCANCEL //NORESTART //ALLUSERS
- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: 3.11
- name: Install Software Renderer
uses: ssciwr/setup-mesa-dist-win@v2
with:
version: '24.2.1'
build-type: 'release-mingw'
- name: Test
run: |
export WEBOTS_DISABLE_SAVE_SCREEN_PERSPECTIVE_ON_CLOSE=true
python scripts/packaging/update_urls.py $(git log -1 --format='%H')
export TESTS_HOME=$PWD # required by cache group in the test suite
export WEBOTS_HOME="/c/Program Files/Webots"
export PATH="${WEBOTS_HOME}/msys64/mingw64/bin:$PATH"
export WEBOTS_HOME_PATH=$PWD # required for tests to find Makefile.include
export BRANCH_HASH=$(git log -1 --format='%H')
export MESA_GL_VERSION_OVERRIDE=3.3
export LIBGL_ALWAYS_SOFTWARE=true
python tests/test_suite.py
- name: Setup tmate session
if: ${{ failure() }}
uses: mxschmitt/action-tmate@v3
with:
detached: true
delete-artifacts:
needs: [build]
needs: [build, test-suite]
if: ${{ always() && !contains(github.event.pull_request.labels.*.name, 'test distribution') && !contains(github.event.pull_request.labels.*.name, 'test webots build') }}
strategy:
matrix:
Expand Down
1 change: 1 addition & 0 deletions projects/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.hdr binary
5 changes: 3 additions & 2 deletions src/webots/gui/WbSingleTaskApplication.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -231,10 +231,11 @@ void WbSingleTaskApplication::showSysInfo() const {
// An OpenGL context is required there for the OpenGL calls like `glGetString`.
// The format is QSurfaceFormat::defaultFormat() => OpenGL 3.3 defined in main.cpp.
QOpenGLContext *context = new QOpenGLContext();
context->create();
if (!context->create())
assert(false);
QOpenGLFunctions *gl = context->functions(); // QOpenGLFunctions_3_3_Core cannot be initialized here on some systems like
// macOS High Sierra and some Ubuntu environments.

assert(gl);
#ifdef _WIN32
const quint32 vendorId = WbSysInfo::gpuVendorId(gl);
const quint32 rendererId = WbSysInfo::gpuDeviceId(gl);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ int main(int argc, char **argv) {
wb_robot_step(2 * TIME_STEP);

// import robot object
FILE *fd = fopen("MyBot.txt", "rb");
FILE *fd = fopen("MyBot.txt", "r");
char contents[4096];
const int n = fread(contents, 1, sizeof(contents), fd);
fclose(fd);
Expand Down
9 changes: 6 additions & 3 deletions tests/test_suite.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ def setupWebots():
command = Command([webotsFullPath, '--sysinfo'])
command.run()
if command.returncode != 0:
raise RuntimeError('Error when getting the Webots information of the system')
raise RuntimeError('Error when getting the Webots information of the system: ' + command.output)
webotsSysInfo = command.output.split('\n')

return webotsFullPath, webotsVersion, webotsSysInfo, webotsEmptyWorldPath
Expand Down Expand Up @@ -181,7 +181,7 @@ def generateWorldsList(groupName):
# generate the list from the arguments
if filesArguments:
for file in filesArguments:
if f'/tests/{groupName}/' in file:
if (os.sep + 'tests' + os.sep + groupName + os.sep) in file:
worldsList.append(file)

# generate the list from 'ls worlds/*.wbt'
Expand All @@ -201,13 +201,16 @@ def generateWorldsList(groupName):
# speaker test not working on github action because of missing sound drivers
# robot window and movie recording test not working on BETA Ubuntu 22.04 GitHub Action environment
# billboard test not working in macos GitHub Action environment
# billboard and robot window not working on windows GitHub Action environment.
if (not filename.endswith('_temp.wbt') and
not ('GITHUB_ACTIONS' in os.environ and (
filename.endswith('speaker.wbt') or
filename.endswith('local_proto_with_texture.wbt') or
(filename.endswith('robot_window_html.wbt') and is_ubuntu_22_04) or
(filename.endswith('supervisor_start_stop_movie.wbt') and is_ubuntu_22_04) or
(filename.endswith('billboard.wbt') and sys.platform == 'darwin')
(filename.endswith('billboard.wbt') and sys.platform == 'darwin') or
(filename.endswith('billboard.wbt') and sys.platform == 'win32') or
(filename.endswith('robot_window_html.wbt') and sys.platform == 'win32')
))):
worldsList.append(filename)

Expand Down
Loading