From fe2879ccc95f40db7a57f3395a811093092e68f2 Mon Sep 17 00:00:00 2001 From: CoolSpy3 <55305038+CoolSpy3@users.noreply.github.com> Date: Mon, 12 Aug 2024 13:05:58 -0700 Subject: [PATCH 01/19] Allow Indirect Field Accesses in Proto Templates (#6614) * allow indirectly referencing proto fields * update other regular expression * add test * fix test worlds * update changelog * use spaces for indentation * update proto headers * run clang-format * enable fields variable with a tag * update regex tests * update docs * update changelog * run clang-format * add curly brackets * fix comment indentation * fix proto name * fix controller name * update the changelog * fix indentation Co-authored-by: Olivier Michel --------- Co-authored-by: Olivier Michel --- docs/reference/changelog-r2024.md | 1 + docs/reference/javascript-procedural-proto.md | 10 +++ docs/reference/lua-procedural-proto.md | 10 +++ src/webots/vrml/WbProtoModel.cpp | 64 +++++++++++-------- src/webots/vrml/WbProtoModel.hpp | 3 +- .../template_indirect_field_access/.gitignore | 1 + .../template_indirect_field_access/Makefile | 19 ++++++ .../template_indirect_field_access.c | 19 ++++++ .../protos/TemplateIndirectFieldAccess.proto | 18 ++++++ .../TemplateIndirectFieldAccessLua.proto | 17 +++++ .../worlds/template_indirect_field_access.wbt | 39 +++++++++++ .../template_indirect_field_access_lua.wbt | 39 +++++++++++ 12 files changed, 213 insertions(+), 27 deletions(-) create mode 100644 tests/protos/controllers/template_indirect_field_access/.gitignore create mode 100644 tests/protos/controllers/template_indirect_field_access/Makefile create mode 100644 tests/protos/controllers/template_indirect_field_access/template_indirect_field_access.c create mode 100644 tests/protos/protos/TemplateIndirectFieldAccess.proto create mode 100644 tests/protos/protos/TemplateIndirectFieldAccessLua.proto create mode 100644 tests/protos/worlds/template_indirect_field_access.wbt create mode 100644 tests/protos/worlds/template_indirect_field_access_lua.wbt diff --git a/docs/reference/changelog-r2024.md b/docs/reference/changelog-r2024.md index 880c9f724b7..10f032d6647 100644 --- a/docs/reference/changelog-r2024.md +++ b/docs/reference/changelog-r2024.md @@ -5,6 +5,7 @@ Released on December **th, 2023. - New Features - **Change the name of the web scene format from `X3D` to `W3D` ([#6280](https://github.com/cyberbotics/webots/pull/6280)).** - Removed support for macOS 11 "Big Sur" and added support for macOS 14 "Sonoma" ([#6580](https://github.com/cyberbotics/webots/pull/6580)). + - Added the `indirectFieldAccess` tag to allow the `fields` variable to be used in proto templates without referencing a specific field ([#6614](https://github.com/cyberbotics/webots/pull/6614)). - Enhancements - Improved the image range of the rotating [Lidar](lidar.md) ([#6324](https://github.com/cyberbotics/webots/pull/6324)). - Cleanup diff --git a/docs/reference/javascript-procedural-proto.md b/docs/reference/javascript-procedural-proto.md index d10426701cd..9bf26da048b 100644 --- a/docs/reference/javascript-procedural-proto.md +++ b/docs/reference/javascript-procedural-proto.md @@ -27,6 +27,8 @@ The first represents the effective value of the field (for instance the one defi - As shown in [this table](#vrml97-type-to-javascript-type-conversion), the conversion of a VRML97 node is an object. This object contains the following keys: "node\_name" containing the VRML97 node name and "fields" which is in turn an object containing the JavaScript representation of the VRML97 node fields. This object is equal to `undefined` if the VRML97 node is not defined (`NULL`). +- By default, the parser only detects fields that are accessed directly in the template statements (i.e. `fields.appearance`). +If you would like to access the `fields` object without referencing a specific field, you can add the following line at the beginning of the PROTO file: `# tags: indirectFieldAccess`. - Objects that are part of [ECMA-262](http://www.ecma-international.org/publications/standards/Ecma-262.htm) are built-in and globally accessible, such as `Math`, `Date` and `String`. - The `context` object provides contextual information about the PROTO. Table [this table](#content-of-the-context-object) shows the available information and the corresponding keys. @@ -823,6 +825,14 @@ The location of this path can be retrieved from the `context` field object, see %end +### PROTO Regeneration +When a field used in a template statement is modified, the PROTO node is regenerated. +This means that the template statements are re-evaluated and the PROTO node is reloaded in the world. +For most nodes, this behavior should not affect the simulation. +However, special care should be taken when using PROTOs that have side effects (e.g. writing to a file). +Additionally, Robot nodes will restart their controllers when regenerated. +Using `tags: indirectFieldAccess` in the PROTO file will cause the PROTO to be regenerated whenever any field is modified, even if it is not used in a template statement. + ### Optimization By default, PROTO files are considered to be deterministic. diff --git a/docs/reference/lua-procedural-proto.md b/docs/reference/lua-procedural-proto.md index 04cbf5ed519..2f1f9aea87a 100644 --- a/docs/reference/lua-procedural-proto.md +++ b/docs/reference/lua-procedural-proto.md @@ -24,6 +24,8 @@ The conversion between the VRML97 types and the Lua types is detailed in [this t This dictionary contains the following keys: "node\_name" containing the VRML97 node name and "fields" which is a dictionary containing the Lua representation of the VRML97 node fields. This dictionary is equal to `nil` if the VRML97 node is not defined (`NULL`). For example, in the SimpleStairs example below, the `fields.appearance.node_name` key contains the `'Appearance'` string. +- By default, the parser only detects fields that are accessed directly in the template statements (i.e. `fields.appearance`). +If you would like to access the `fields` object without referencing a specific field, you can add the following line at the beginning of the PROTO file: `# tags: indirectFieldAccess`. - The `context` dictionary provides contextual information about the PROTO. Table [this table](#content-of-the-context-dictionary) shows the available information and its corresponding keys. - The VRML97 comment ("#") prevails over the Lua statements. @@ -88,6 +90,14 @@ The following standard fonts are available to write on the texture: In addition to these fonts, it is possible to add other TrueType fonts file in your `PROJECT_HOME/fonts` directory. +### PROTO Regeneration +When a field used in a template statement is modified, the PROTO node is regenerated. +This means that the template statements are re-evaluated and the PROTO node is reloaded in the world. +For most nodes, this behavior should not affect the simulation. +However, special care should be taken when using PROTOs that have side effects (e.g. writing to a file). +Additionally, Robot nodes will restart their controllers when regenerated. +Using `tags: indirectFieldAccess` in the PROTO file will cause the PROTO to be regenerated whenever any field is modified, even if it is not used in a template statement. + ### Optimization By default, PROTO files are considered to be deterministic. diff --git a/src/webots/vrml/WbProtoModel.cpp b/src/webots/vrml/WbProtoModel.cpp index 4a2a13cccc8..07de98d3812 100644 --- a/src/webots/vrml/WbProtoModel.cpp +++ b/src/webots/vrml/WbProtoModel.cpp @@ -69,6 +69,7 @@ WbProtoModel::WbProtoModel(WbTokenizer *tokenizer, const QString &worldPath, con mDocumentationUrl = tokenizer->documentationUrl(); mTemplateLanguage = tokenizer->templateLanguage(); mIsDeterministic = !mTags.contains("nonDeterministic"); + mHasIndirectFieldAccess = mTags.contains("indirectFieldAccess"); WbParser parser(tokenizer); while (tokenizer->peekWord() == "EXTERNPROTO" || tokenizer->peekWord() == "IMPORTABLE") // consume EXTERNPROTO declarations @@ -209,15 +210,23 @@ WbProtoModel::WbProtoModel(WbTokenizer *tokenizer, const QString &worldPath, con previousToken = token; token = tokenizer->nextToken(); + if (mHasIndirectFieldAccess) { + foreach (WbFieldModel *model, mFieldModels) + model->setTemplateRegenerator(true); + } + if (token->isTemplateStatement()) { mTemplate = true; - foreach (WbFieldModel *model, mFieldModels) { - // condition explanation: if (token contains modelName and not a Lua identifier containing modelName such as - // "my_awesome_modelName") - if (token->word().contains(QRegularExpression( - QString("(^|[^a-zA-Z0-9_])fields\\.%1($|[^a-zA-Z0-9_])").arg(QRegularExpression::escape(model->name()))))) { - model->setTemplateRegenerator(true); + if (!mHasIndirectFieldAccess) { // If the proto has indirect field access, we've already set the fields as template + // regenerators + foreach (WbFieldModel *model, mFieldModels) { + // condition explanation: if (token contains modelName and not a Lua identifier containing modelName such as + // "my_awesome_modelName") or (token contains fields and not a Lua identifier containing fields such as "my_fields") + if (token->word().contains( + QRegularExpression(QString("(^|\\W)fields\\.%1($|\\W)").arg(QRegularExpression::escape(model->name()))))) { + model->setTemplateRegenerator(true); + } } } } else if (readBaseType) { @@ -274,26 +283,29 @@ WbProtoModel::WbProtoModel(WbTokenizer *tokenizer, const QString &worldPath, con throw 0; } } else if (token->isString()) { - // check which parameter need to regenerate the template instance from inside a string - foreach (WbFieldModel *model, mFieldModels) { - // regex test cases: - // "You know nothing, John Snow." => false - // "%{=fields.model->name()}%" => false - // "%{= fields.model->name().value.x }% %{= fields.model->name().value.y }%" => true - // "abc %{= fields.model->name().value.y }% def" => true - // "%{= 17 % fields.model->name().value.y * 88 }%" => true - // "fields.model->name().value.y" => false - // "%{}% fields.model->name().value.y %{}%" => false - // "%{ a = \"fields.model->name().value.y\" }%" => false - // "%{= \"fields.model->name().value.y\" }%" => false - // "%{= fields.model->name().value.y }%" => true - if (token->word().contains(QRegularExpression(QString("%1(?:(?!%2|\").)*fields\\.%3(?:(?!%4|\").)*%5") - .arg(open) - .arg(close) - .arg(QRegularExpression::escape(model->name())) - .arg(close) - .arg(close)))) - model->setTemplateRegenerator(true); + if (!mHasIndirectFieldAccess) { // If the proto has indirect field access, we've already set the fields as template + // regenerators + // check which parameter need to regenerate the template instance from inside a string + foreach (WbFieldModel *model, mFieldModels) { + // regex test cases: + // "You know nothing, John Snow." => false + // "%{=fields.model->name()}%" => true + // "%{= fields.model->name().value.x }% %{= fields.model->name().value.y }%" => true + // "abc %{= fields.model->name().value.y }% def" => true + // "%{= 17 % fields.model->name().value.y * 88 }%" => true + // "fields.model->name().value.y" => false + // "%{}% fields.model->name().value.y %{}%" => false + // "%{ a = \"fields.model->name().value.y\" }%" => false + // "%{= \"fields.model->name().value.y\" }%" => false + // "%{= fields.model->name().value.y }%" => true + if (token->word().contains(QRegularExpression(QString("%1(?:(?!%2|\").)*fields\\.%3(?:(?!%4|\").)*%5") + .arg(open) + .arg(close) + .arg(QRegularExpression::escape(model->name())) + .arg(close) + .arg(close)))) + model->setTemplateRegenerator(true); + } } } diff --git a/src/webots/vrml/WbProtoModel.hpp b/src/webots/vrml/WbProtoModel.hpp index dccf030bd97..49fca7c8971 100644 --- a/src/webots/vrml/WbProtoModel.hpp +++ b/src/webots/vrml/WbProtoModel.hpp @@ -125,7 +125,8 @@ class WbProtoModel : public QObject { WbVersion mFileVersion; QString mName; QString mInfo; - bool mIsDeterministic; // i.e doesn't have the 'nonDeterministic' tag + bool mIsDeterministic; // i.e doesn't have the 'nonDeterministic' tag + bool mHasIndirectFieldAccess; // i.e. has the 'indirectFieldAccess' tag QList mFieldModels; QString mUrl; // how the PROTO is referenced diff --git a/tests/protos/controllers/template_indirect_field_access/.gitignore b/tests/protos/controllers/template_indirect_field_access/.gitignore new file mode 100644 index 00000000000..7abe0a8a413 --- /dev/null +++ b/tests/protos/controllers/template_indirect_field_access/.gitignore @@ -0,0 +1 @@ +/template_indirect_field_access diff --git a/tests/protos/controllers/template_indirect_field_access/Makefile b/tests/protos/controllers/template_indirect_field_access/Makefile new file mode 100644 index 00000000000..9734e04d8e9 --- /dev/null +++ b/tests/protos/controllers/template_indirect_field_access/Makefile @@ -0,0 +1,19 @@ +# Copyright 1996-2023 Cyberbotics Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +### Do not modify: this includes Webots global Makefile.include +null := +space := $(null) $(null) +WEBOTS_HOME_PATH?=$(subst $(space),\ ,$(strip $(subst \,/,$(WEBOTS_HOME)))) +include $(WEBOTS_HOME_PATH)/resources/Makefile.include diff --git a/tests/protos/controllers/template_indirect_field_access/template_indirect_field_access.c b/tests/protos/controllers/template_indirect_field_access/template_indirect_field_access.c new file mode 100644 index 00000000000..6a5605d5473 --- /dev/null +++ b/tests/protos/controllers/template_indirect_field_access/template_indirect_field_access.c @@ -0,0 +1,19 @@ +#include +#include +#include + +#include "../../../lib/ts_assertion.h" +#include "../../../lib/ts_utils.h" + +#define TIME_STEP 32 + +int main(int argc, char **argv) { + ts_setup(argv[1]); // give the controller args + WbNodeRef proto = wb_supervisor_node_get_from_def("PROTO_template_indirect_field_access"); + WbFieldRef radarCrossSection = wb_supervisor_node_get_proto_field(proto, "radarCrossSection"); + ts_assert_double_in_delta(wb_supervisor_field_get_sf_float(radarCrossSection), 6.0, 0.0001, + "radarCrossSection should be 6.0"); + ts_send_success(); + + return EXIT_SUCCESS; +} diff --git a/tests/protos/protos/TemplateIndirectFieldAccess.proto b/tests/protos/protos/TemplateIndirectFieldAccess.proto new file mode 100644 index 00000000000..b115bed1463 --- /dev/null +++ b/tests/protos/protos/TemplateIndirectFieldAccess.proto @@ -0,0 +1,18 @@ +#VRML_SIM R2024a utf8 +# tags: indirectFieldAccess +# template language: javascript + +PROTO TemplateIndirectFieldAccess [ + unconnectedField SFFloat field1 1 + unconnectedField SFFloat field2 5 +] +{ + Solid { + %< + function sum(fieldsObj) { + return fieldsObj.field1.value + fieldsObj.field2.value; + } + >% + radarCrossSection %<= sum(fields) >% + } +} diff --git a/tests/protos/protos/TemplateIndirectFieldAccessLua.proto b/tests/protos/protos/TemplateIndirectFieldAccessLua.proto new file mode 100644 index 00000000000..d6b9a0c20e3 --- /dev/null +++ b/tests/protos/protos/TemplateIndirectFieldAccessLua.proto @@ -0,0 +1,17 @@ +#VRML_SIM R2024a utf8 +# tags: indirectFieldAccess + +PROTO TemplateIndirectFieldAccessLua [ + unconnectedField SFFloat field1 1 + unconnectedField SFFloat field2 5 +] +{ + Solid { + %{ + function sum(fieldsObj) + return fieldsObj.field1.value + fieldsObj.field2.value; + end + }% + radarCrossSection %{= sum(fields) }% + } +} diff --git a/tests/protos/worlds/template_indirect_field_access.wbt b/tests/protos/worlds/template_indirect_field_access.wbt new file mode 100644 index 00000000000..1d79b67a155 --- /dev/null +++ b/tests/protos/worlds/template_indirect_field_access.wbt @@ -0,0 +1,39 @@ +#VRML_SIM R2024a utf8 + +EXTERNPROTO "webots://tests/protos/protos/TemplateIndirectFieldAccess.proto" +EXTERNPROTO "webots://tests/default/protos/TestSuiteEmitter.proto" +EXTERNPROTO "webots://tests/default/protos/TestSuiteSupervisor.proto" + +WorldInfo { + coordinateSystem "NUE" + lineScale 1 +} +Viewpoint { + orientation -0.8999515417256047 0.42450878383113183 0.0993957493855782 0.509061 + position 0.303563 0.46348 0.936228 +} +Background { + skyColor [ + 0 0.811765 0.992157 + ] +} +DirectionalLight { + direction 0 0.5 -1 +} +Robot { + translation 0.1 0 0.15 + rotation 0 1 0 1.5708 + children [ + DEF PROTO_template_indirect_field_access TemplateIndirectFieldAccess { + } + TestSuiteEmitter { + } + ] + controller "template_indirect_field_access" + controllerArgs [ + "template_indirect_field_access" + ] + supervisor TRUE +} +TestSuiteSupervisor { +} diff --git a/tests/protos/worlds/template_indirect_field_access_lua.wbt b/tests/protos/worlds/template_indirect_field_access_lua.wbt new file mode 100644 index 00000000000..b16b039f496 --- /dev/null +++ b/tests/protos/worlds/template_indirect_field_access_lua.wbt @@ -0,0 +1,39 @@ +#VRML_SIM R2024a utf8 + +EXTERNPROTO "webots://tests/protos/protos/TemplateIndirectFieldAccessLua.proto" +EXTERNPROTO "webots://tests/default/protos/TestSuiteEmitter.proto" +EXTERNPROTO "webots://tests/default/protos/TestSuiteSupervisor.proto" + +WorldInfo { + coordinateSystem "NUE" + lineScale 1 +} +Viewpoint { + orientation -0.8999515417256047 0.42450878383113183 0.0993957493855782 0.509061 + position 0.303563 0.46348 0.936228 +} +Background { + skyColor [ + 0 0.811765 0.992157 + ] +} +DirectionalLight { + direction 0 0.5 -1 +} +Robot { + translation 0.1 0 0.15 + rotation 0 1 0 1.5708 + children [ + DEF PROTO_template_indirect_field_access TemplateIndirectFieldAccessLua { + } + TestSuiteEmitter { + } + ] + controller "template_indirect_field_access" + controllerArgs [ + "template_indirect_field_access (lua)" + ] + supervisor TRUE +} +TestSuiteSupervisor { +} From 0bcfb17c42b23e97645484ca8183817924ad501c Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Wed, 14 Aug 2024 09:37:29 -0700 Subject: [PATCH 02/19] Fix incomplete loading while minimized under some windowing systems. (#6617) * Fix incomplete loading under some windowing systems. * Add changelog entry. * Apply suggestions from code review Co-authored-by: Olivier Michel * Revert "Apply suggestions from code review" This reverts commit 7804a09cb772339c93687de95e333441e2f7ab43. * Revert "Fix incomplete loading under some windowing systems." This reverts commit 0d9dbb29d6ffb8cdc1dc675c6c26b444a6d94eed. * Rework approach to ensure that only offscreen framebuffers are used for the initial render. --------- Co-authored-by: Olivier Michel --- docs/reference/changelog-r2024.md | 1 + include/wren/scene.h | 5 +++-- src/webots/gui/WbView3D.cpp | 9 +++++---- src/webots/gui/WbView3D.hpp | 2 +- src/webots/gui/WbWrenWindow.cpp | 9 +++++---- src/webots/gui/WbWrenWindow.hpp | 2 +- .../nodes/utils/WbVirtualRealityHeadset.cpp | 2 +- src/webots/wren/WbWrenCamera.cpp | 2 +- src/webots/wren/WbWrenPicker.cpp | 2 +- src/wren/Scene.cpp | 17 +++++++++-------- src/wren/Scene.hpp | 4 ++-- src/wren/Viewport.cpp | 11 ----------- src/wren/demo/default.cpp | 2 +- src/wren/demo/phong.cpp | 2 +- 14 files changed, 32 insertions(+), 38 deletions(-) diff --git a/docs/reference/changelog-r2024.md b/docs/reference/changelog-r2024.md index 10f032d6647..54add51c1ef 100644 --- a/docs/reference/changelog-r2024.md +++ b/docs/reference/changelog-r2024.md @@ -20,4 +20,5 @@ Released on December **th, 2023. - Fixed invalid absolute sound file path resulted in crash ([#6593](https://github.com/cyberbotics/webots/pull/6593)) - Fixed Speaker relative sound file path not working with external controller ([#6605](https://github.com/cyberbotics/webots/pull/6605)). - Fixed the bug that when the language is Python, getTargets() cannot correctly obtain the multi-target data detected by the radar ([#6606](https://github.com/cyberbotics/webots/pull/6606)) + - Fixed incomplete loading while minimized under some windowing systems ([#6617](https://github.com/cyberbotics/webots/pull/6617)). - Fixed unitialized sliding friction when using asymmetric rolling friction ([#6618](https://github.com/cyberbotics/webots/pull/6618)). diff --git a/include/wren/scene.h b/include/wren/scene.h index 9788bbb2935..ea0d7fc1d0d 100644 --- a/include/wren/scene.h +++ b/include/wren/scene.h @@ -42,8 +42,9 @@ void wr_scene_apply_pending_updates(WrScene *scene); /* The 'materialName' parameter is optional (can be set to NULL for default), and if set, force the renderables to use the named material */ -void wr_scene_render(WrScene *scene, const char *material_name, bool culling); -void wr_scene_render_to_viewports(WrScene *scene, int count, WrViewport **viewports, const char *material_name, bool culling); +void wr_scene_render(WrScene *scene, const char *material_name, bool culling, bool offScreen); +void wr_scene_render_to_viewports(WrScene *scene, int count, WrViewport **viewports, const char *material_name, bool culling, + bool offScreen); void wr_scene_set_ambient_light(const float *ambient_light); diff --git a/src/webots/gui/WbView3D.cpp b/src/webots/gui/WbView3D.cpp index f7ce3bb2849..b60cc9bd591 100644 --- a/src/webots/gui/WbView3D.cpp +++ b/src/webots/gui/WbView3D.cpp @@ -1111,8 +1111,9 @@ void WbView3D::setWorld(WbSimulationWorld *w) { WbWrenOpenGlContext::doneWren(); - // first rendering without culling to make sure every meshes/textures are actually loaded on the GPU - renderNow(false); + // first rendering is offscreen without culling to make sure every meshes/textures are actually + // loaded on the GPU + renderNow(false, true); } void WbView3D::restoreOptionalRendering(const QStringList &enabledCenterOfMassNodeNames, @@ -1441,7 +1442,7 @@ void WbView3D::resizeWren(int width, int height) { emit resized(); } -void WbView3D::renderNow(bool culling) { +void WbView3D::renderNow(bool culling, bool offScreen) { if (!wr_gl_state_is_initialized()) initialize(); @@ -1461,7 +1462,7 @@ void WbView3D::renderNow(bool culling) { WbWrenOpenGlContext::doneWren(); } else #endif - WbWrenWindow::renderNow(culling); + WbWrenWindow::renderNow(culling, offScreen); mLastRefreshTimer.start(); emit mainRenderingEnded(mPhysicsRefresh); diff --git a/src/webots/gui/WbView3D.hpp b/src/webots/gui/WbView3D.hpp index d81d292a6c8..be3dec0bf64 100644 --- a/src/webots/gui/WbView3D.hpp +++ b/src/webots/gui/WbView3D.hpp @@ -97,7 +97,7 @@ public slots: protected slots: // cppcheck-suppress virtualCallInConstructor - void renderNow(bool culling = true) override; + void renderNow(bool culling = true, bool offScreen = false) override; protected: void initialize() override; diff --git a/src/webots/gui/WbWrenWindow.cpp b/src/webots/gui/WbWrenWindow.cpp index 0d404cd7131..44960f92495 100644 --- a/src/webots/gui/WbWrenWindow.cpp +++ b/src/webots/gui/WbWrenWindow.cpp @@ -200,8 +200,8 @@ void WbWrenWindow::renderLater() { } } -void WbWrenWindow::renderNow(bool culling) { - if (!isExposed() || !wr_gl_state_is_initialized()) +void WbWrenWindow::renderNow(bool culling, bool offScreen) { + if ((!isExposed() && !offScreen) || !wr_gl_state_is_initialized()) return; static int first = true; @@ -225,9 +225,10 @@ void WbWrenWindow::renderNow(bool culling) { WbWrenOpenGlContext::makeWrenCurrent(); - wr_scene_render(wr_scene_get_instance(), NULL, culling); + wr_scene_render(wr_scene_get_instance(), NULL, culling, offScreen); - WbWrenOpenGlContext::instance()->swapBuffers(this); + if (!offScreen) + WbWrenOpenGlContext::instance()->swapBuffers(this); WbWrenOpenGlContext::doneWren(); if (mVideoStreamingServer && mVideoStreamingServer->isNewFrameNeeded() && !first) diff --git a/src/webots/gui/WbWrenWindow.hpp b/src/webots/gui/WbWrenWindow.hpp index 9a49b17efe4..dee3c098876 100644 --- a/src/webots/gui/WbWrenWindow.hpp +++ b/src/webots/gui/WbWrenWindow.hpp @@ -65,7 +65,7 @@ public slots: protected: virtual void initialize(); - virtual void renderNow(bool culling = true); + virtual void renderNow(bool culling = true, bool offScreen = false); virtual void resizeWren(int width, int height); bool event(QEvent *event) override; diff --git a/src/webots/nodes/utils/WbVirtualRealityHeadset.cpp b/src/webots/nodes/utils/WbVirtualRealityHeadset.cpp index 07f7653fce8..9beeeb7bbad 100644 --- a/src/webots/nodes/utils/WbVirtualRealityHeadset.cpp +++ b/src/webots/nodes/utils/WbVirtualRealityHeadset.cpp @@ -406,7 +406,7 @@ void WbVirtualRealityHeadset::updateOrientationAndPosition() { if (mWrenViewports[0] && mSystem) { WbWrenOpenGlContext::makeWrenCurrent(); - wr_scene_render_to_viewports(wr_scene_get_instance(), 2, mWrenViewports, NULL, true); + wr_scene_render_to_viewports(wr_scene_get_instance(), 2, mWrenViewports, NULL, true, false); WbWrenOpenGlContext::doneWren(); vr::VRCompositor()->Submit(vr::Eye_Left, mTextureReferences[LEFT], mTextureBounds); vr::VRCompositor()->Submit(vr::Eye_Right, mTextureReferences[RIGHT], mTextureBounds); diff --git a/src/webots/wren/WbWrenCamera.cpp b/src/webots/wren/WbWrenCamera.cpp index f507904b2d1..eedf0c47c48 100644 --- a/src/webots/wren/WbWrenCamera.cpp +++ b/src/webots/wren/WbWrenCamera.cpp @@ -452,7 +452,7 @@ void WbWrenCamera::render() { else if (mType == 's') materialName = "segmentation"; wr_scene_enable_depth_reset(wr_scene_get_instance(), false); - wr_scene_render_to_viewports(wr_scene_get_instance(), numActiveViewports, mViewportsToRender, materialName, true); + wr_scene_render_to_viewports(wr_scene_get_instance(), numActiveViewports, mViewportsToRender, materialName, true, false); if (!isPlanarProjection()) applySphericalPostProcessingEffect(); diff --git a/src/webots/wren/WbWrenPicker.cpp b/src/webots/wren/WbWrenPicker.cpp index 2b78ab90c75..742051aecc8 100644 --- a/src/webots/wren/WbWrenPicker.cpp +++ b/src/webots/wren/WbWrenPicker.cpp @@ -134,7 +134,7 @@ bool WbWrenPicker::pick(int x, int y) { wr_viewport_enable_skybox(mViewport, false); wr_scene_enable_translucence(scene, false); wr_scene_enable_depth_reset(scene, false); - wr_scene_render_to_viewports(scene, 1, &mViewport, "picking", true); + wr_scene_render_to_viewports(scene, 1, &mViewport, "picking", true, false); wr_scene_enable_depth_reset(scene, true); wr_viewport_enable_skybox(mViewport, true); wr_scene_enable_translucence(scene, true); diff --git a/src/wren/Scene.cpp b/src/wren/Scene.cpp index 90791423099..87258a3e4dc 100644 --- a/src/wren/Scene.cpp +++ b/src/wren/Scene.cpp @@ -224,7 +224,7 @@ namespace wren { debug::printSceneTree(); } - void Scene::render(bool culling) { + void Scene::render(bool culling, bool offScreen) { assert(glstate::isInitialized()); ++mFrameCounter; @@ -233,10 +233,10 @@ namespace wren { // debug::printCacheContents(); // debug::printSceneTree(); - renderToViewports({mMainViewport}, culling); + renderToViewports({mMainViewport}, culling, offScreen); } - void Scene::renderToViewports(const std::vector &viewports, bool culling) { + void Scene::renderToViewports(const std::vector &viewports, bool culling, bool offScreen) { assert(glstate::isInitialized()); DEBUG("Notify frame listeners..."); @@ -282,7 +282,7 @@ namespace wren { } } else { renderToViewport(culling); - if (mCurrentViewport == mMainViewport && mCurrentViewport->frameBuffer()) { + if (!offScreen && mCurrentViewport == mMainViewport && mCurrentViewport->frameBuffer()) { glstate::bindDrawFrameBuffer(0); mCurrentViewport->frameBuffer()->blit(0, true, false, false, 0, 0, 0, 0, 0, 0, mCurrentViewport->width() * mCurrentViewport->pixelRatio(), @@ -909,22 +909,23 @@ void wr_scene_terminate_frame_capture(WrScene *scene) { reinterpret_cast(scene)->terminateFrameCapture(); } -void wr_scene_render(WrScene *scene, const char *material_name, bool culling) { +void wr_scene_render(WrScene *scene, const char *material_name, bool culling, bool offScreen) { if (material_name) wren::Renderable::setUseMaterial(material_name); - reinterpret_cast(scene)->render(culling); + reinterpret_cast(scene)->render(culling, offScreen); wren::Renderable::setUseMaterial(NULL); } -void wr_scene_render_to_viewports(WrScene *scene, int count, WrViewport **viewports, const char *material_name, bool culling) { +void wr_scene_render_to_viewports(WrScene *scene, int count, WrViewport **viewports, const char *material_name, bool culling, + bool offScreen) { if (material_name) wren::Renderable::setUseMaterial(material_name); wren::Viewport **start = reinterpret_cast(viewports); std::vector viewportsVector(start, start + count); - reinterpret_cast(scene)->renderToViewports(viewportsVector, culling); + reinterpret_cast(scene)->renderToViewports(viewportsVector, culling, offScreen); wren::Renderable::setUseMaterial(NULL); } diff --git a/src/wren/Scene.hpp b/src/wren/Scene.hpp index d865cbd9043..9f68654b77e 100644 --- a/src/wren/Scene.hpp +++ b/src/wren/Scene.hpp @@ -90,8 +90,8 @@ namespace wren { int computeNodeCount() const; static void printSceneTree(); - void render(bool culling); - void renderToViewports(const std::vector &viewports, bool culling); + void render(bool culling, bool offScreen = false); + void renderToViewports(const std::vector &viewports, bool culling, bool offScreen = false); void addFrameListener(void (*listener)()) { mListeners.push_back(listener); } void removeFrameListener(void (*listener)()); diff --git a/src/wren/Viewport.cpp b/src/wren/Viewport.cpp index 1d5e25fdeeb..723477ebb10 100644 --- a/src/wren/Viewport.cpp +++ b/src/wren/Viewport.cpp @@ -180,10 +180,6 @@ namespace wren { } } - glstate::bindDrawFrameBuffer(0); - - clear(); - for (size_t i = 0; i < mPostProcessingEffects.size(); ++i) mPostProcessingEffects[i]->apply(); } @@ -202,9 +198,6 @@ namespace wren { mAmbientOcclusionEffect->setResultFrameBuffer(mFrameBuffer); mAmbientOcclusionEffect->firstPass()->setInputTexture(0, mFrameBuffer->outputTexture(0)); } - glstate::bindDrawFrameBuffer(0); - - clear(); mAmbientOcclusionEffect->apply(); } @@ -224,10 +217,6 @@ namespace wren { mAntiAliasingEffect->firstPass()->setInputTexture(0, mFrameBuffer->outputTexture(0)); } - glstate::bindDrawFrameBuffer(0); - - clear(); - mAntiAliasingEffect->apply(); } } diff --git a/src/wren/demo/default.cpp b/src/wren/demo/default.cpp index fad345389c1..1578317ef93 100644 --- a/src/wren/demo/default.cpp +++ b/src/wren/demo/default.cpp @@ -99,7 +99,7 @@ static void create_wren_scene() { // Render function. static void render() { - wr_scene_render(wr_scene_get_instance(), NULL, true); + wr_scene_render(wr_scene_get_instance(), NULL, true, false); glutSwapBuffers(); } diff --git a/src/wren/demo/phong.cpp b/src/wren/demo/phong.cpp index 7ed9807a5c3..f7facb5ce95 100644 --- a/src/wren/demo/phong.cpp +++ b/src/wren/demo/phong.cpp @@ -166,7 +166,7 @@ static void render() { static int i = 0; printf("\33[2K\rrendering iteration %d", i++); fflush(stdout); - wr_scene_render(wr_scene_get_instance(), NULL, true); + wr_scene_render(wr_scene_get_instance(), NULL, true, false); glutSwapBuffers(); } From 0ee25809535c09667f0ecd0b275645d2ab07c317 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Thu, 15 Aug 2024 11:35:24 -0700 Subject: [PATCH 03/19] Make supervisor_export_image use FBO instead of screen framebuffer. (#6622) * Make supervisor_export_image use FBO instead of screen framebuffer. This makes the supervisor_export_image test pass under Wayland and has the added benefit of making that test (and the billboard test) work with "--no-rendering --minimize". As a result those tests are moved into the api group and the with_rendering group is removed. * Add changelog entry. --- docs/reference/changelog-r2024.md | 1 + include/wren/scene.h | 3 +-- src/webots/gui/WbView3D.cpp | 2 +- src/webots/gui/WbWrenWindow.cpp | 12 +--------- src/wren/Scene.cpp | 14 +++++------ src/wren/Scene.hpp | 3 +-- tests/Makefile | 7 ++---- .../controllers/billboard/.gitignore | 0 .../controllers/billboard/Makefile | 0 .../controllers/billboard/billboard.c | 0 .../supervisor_export_image/.gitignore | 0 .../supervisor_export_image/Makefile | 0 .../supervisor_export_image.c | 0 .../worlds/billboard.wbt | 0 .../worlds/supervisor_export_image.wbt | 0 tests/test_suite.py | 8 +++---- tests/with_rendering/.gitignore | 2 -- tests/with_rendering/Makefile | 16 ------------- tests/with_rendering/controllers/.gitignore | 2 -- tests/with_rendering/controllers/Makefile | 24 ------------------- tests/with_rendering/worlds/.gitignore | 2 -- .../worlds/.supervisor_export_image.wbproj | 12 ---------- 22 files changed, 16 insertions(+), 92 deletions(-) rename tests/{with_rendering => api}/controllers/billboard/.gitignore (100%) rename tests/{with_rendering => api}/controllers/billboard/Makefile (100%) rename tests/{with_rendering => api}/controllers/billboard/billboard.c (100%) rename tests/{with_rendering => api}/controllers/supervisor_export_image/.gitignore (100%) rename tests/{with_rendering => api}/controllers/supervisor_export_image/Makefile (100%) rename tests/{with_rendering => api}/controllers/supervisor_export_image/supervisor_export_image.c (100%) rename tests/{with_rendering => api}/worlds/billboard.wbt (100%) rename tests/{with_rendering => api}/worlds/supervisor_export_image.wbt (100%) delete mode 100644 tests/with_rendering/.gitignore delete mode 100644 tests/with_rendering/Makefile delete mode 100644 tests/with_rendering/controllers/.gitignore delete mode 100644 tests/with_rendering/controllers/Makefile delete mode 100644 tests/with_rendering/worlds/.gitignore delete mode 100644 tests/with_rendering/worlds/.supervisor_export_image.wbproj diff --git a/docs/reference/changelog-r2024.md b/docs/reference/changelog-r2024.md index 54add51c1ef..91cf19d8809 100644 --- a/docs/reference/changelog-r2024.md +++ b/docs/reference/changelog-r2024.md @@ -22,3 +22,4 @@ Released on December **th, 2023. - Fixed the bug that when the language is Python, getTargets() cannot correctly obtain the multi-target data detected by the radar ([#6606](https://github.com/cyberbotics/webots/pull/6606)) - Fixed incomplete loading while minimized under some windowing systems ([#6617](https://github.com/cyberbotics/webots/pull/6617)). - Fixed unitialized sliding friction when using asymmetric rolling friction ([#6618](https://github.com/cyberbotics/webots/pull/6618)). + - Made `supervisor_export_image` work even if using the `--minimize --no-rendering` options ([#6622](https://github.com/cyberbotics/webots/pull/6622)). diff --git a/include/wren/scene.h b/include/wren/scene.h index ea0d7fc1d0d..ec9fd2e4f27 100644 --- a/include/wren/scene.h +++ b/include/wren/scene.h @@ -68,8 +68,7 @@ void wr_scene_enable_depth_reset(WrScene *scene, bool enable); void wr_scene_add_frame_listener(WrScene *scene, void (*listener)()); void wr_scene_remove_frame_listener(WrScene *scene, void (*listener)()); -void wr_scene_get_main_buffer(WrScene *scene, int width, int height, unsigned int format, unsigned int data_type, - unsigned int buffer_type, void *buffer); +void wr_scene_get_main_buffer(int width, int height, unsigned int format, unsigned int data_type, void *buffer); void wr_scene_init_frame_capture(WrScene *scene, int pixel_buffer_count, unsigned int *pixel_buffer_ids, int frame_size); void wr_scene_bind_pixel_buffer(WrScene *scene, int buffer); diff --git a/src/webots/gui/WbView3D.cpp b/src/webots/gui/WbView3D.cpp index b60cc9bd591..5728028f55e 100644 --- a/src/webots/gui/WbView3D.cpp +++ b/src/webots/gui/WbView3D.cpp @@ -332,7 +332,7 @@ void WbView3D::refresh() { const WbSimulationState *const sim = WbSimulationState::instance(); mPhysicsRefresh = true; if (mScreenshotRequested) - renderNow(); + renderNow(true, true); else if (sim->isPaused()) renderLater(); else if (WbVideoRecorder::instance()->isRecording()) { diff --git a/src/webots/gui/WbWrenWindow.cpp b/src/webots/gui/WbWrenWindow.cpp index 44960f92495..dba767bb7e8 100644 --- a/src/webots/gui/WbWrenWindow.cpp +++ b/src/webots/gui/WbWrenWindow.cpp @@ -446,15 +446,5 @@ void WbWrenWindow::feedMultimediaStreamer() { } void WbWrenWindow::readPixels(int width, int height, unsigned int format, void *buffer) { -#ifdef __linux__ - if (WbSysInfo::isVirtualMachine()) { - // Reading the front buffer is not supported by all OpenGL implementations (especially on Linux running in a VM). - // In that case, to read the front buffer, we need to swap the buffers, read the back buffer and swap the buffers again. - // However, doing this may cause flickering on platforms where reading the front buffer is supported (including macOS). - WbWrenOpenGlContext::instance()->swapBuffers(this); - wr_scene_get_main_buffer(wr_scene_get_instance(), width, height, format, GL_UNSIGNED_BYTE, GL_BACK, buffer); - WbWrenOpenGlContext::instance()->swapBuffers(this); - } else -#endif - wr_scene_get_main_buffer(wr_scene_get_instance(), width, height, format, GL_UNSIGNED_BYTE, GL_FRONT, buffer); + wr_scene_get_main_buffer(width, height, format, GL_UNSIGNED_BYTE, buffer); } diff --git a/src/wren/Scene.cpp b/src/wren/Scene.cpp index 87258a3e4dc..913277bb3a5 100644 --- a/src/wren/Scene.cpp +++ b/src/wren/Scene.cpp @@ -101,11 +101,10 @@ namespace wren { assert(!Texture2d::cachedItemCount()); } - void Scene::getMainBuffer(int width, int height, unsigned int format, unsigned int data_type, unsigned int buffer_type, - void *buffer) { - assert(buffer_type == GL_FRONT || buffer_type == GL_BACK); - glstate::bindFrameBuffer(0); - glReadBuffer(buffer_type); + void Scene::getMainBuffer(int width, int height, unsigned int format, unsigned int data_type, void *buffer) { + glstate::bindFrameBuffer(instance()->mMainViewport->frameBuffer()->glName()); + glReadBuffer(GL_COLOR_ATTACHMENT0); + glFinish(); glReadPixels(0, 0, width, height, format, data_type, buffer); } @@ -884,9 +883,8 @@ void wr_scene_apply_pending_updates(WrScene *scene) { reinterpret_cast(scene)->applyPendingUpdates(); } -void wr_scene_get_main_buffer(WrScene *scene, int width, int height, unsigned int format, unsigned int data_type, - unsigned int buffer_type, void *buffer) { - reinterpret_cast(scene)->getMainBuffer(width, height, format, data_type, buffer_type, buffer); +void wr_scene_get_main_buffer(int width, int height, unsigned int format, unsigned int data_type, void *buffer) { + wren::Scene::getMainBuffer(width, height, format, data_type, buffer); } void wr_scene_init_frame_capture(WrScene *scene, int pixel_buffer_count, unsigned int *pixel_buffer_ids, int frame_size) { diff --git a/src/wren/Scene.hpp b/src/wren/Scene.hpp index 9f68654b77e..5e8b4ea032b 100644 --- a/src/wren/Scene.hpp +++ b/src/wren/Scene.hpp @@ -53,8 +53,7 @@ namespace wren { void reset(); static void applyPendingUpdates(); - static void getMainBuffer(int width, int height, unsigned int format, unsigned int data_type, unsigned int buffer_type, - void *buffer); + static void getMainBuffer(int width, int height, unsigned int format, unsigned int data_type, void *buffer); void initFrameCapture(int pixelBufferCount, unsigned int *pixelBufferIds, int frameSize); static void bindPixelBuffer(int buffer); void *mapPixelBuffer(unsigned int accessMode); diff --git a/tests/Makefile b/tests/Makefile index 3f5ddfafe0f..523b2988eb4 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -12,9 +12,9 @@ # See the License for the specific language governing permissions and # limitations under the License. -.PHONY: api cache other_api parser physics protos rendering with_rendering +.PHONY: api cache other_api parser physics protos rendering -release debug profile: api cache other_api parser physics protos rendering with_rendering +release debug profile: api cache other_api parser physics protos rendering clean: api cache parser physics protos rendering @-rm -rf *.txt \ @@ -56,6 +56,3 @@ protos: rendering: +@make -s -C rendering $(MAKECMDGOALS) - -with_rendering: - +@make -s -C with_rendering $(MAKECMDGOALS) diff --git a/tests/with_rendering/controllers/billboard/.gitignore b/tests/api/controllers/billboard/.gitignore similarity index 100% rename from tests/with_rendering/controllers/billboard/.gitignore rename to tests/api/controllers/billboard/.gitignore diff --git a/tests/with_rendering/controllers/billboard/Makefile b/tests/api/controllers/billboard/Makefile similarity index 100% rename from tests/with_rendering/controllers/billboard/Makefile rename to tests/api/controllers/billboard/Makefile diff --git a/tests/with_rendering/controllers/billboard/billboard.c b/tests/api/controllers/billboard/billboard.c similarity index 100% rename from tests/with_rendering/controllers/billboard/billboard.c rename to tests/api/controllers/billboard/billboard.c diff --git a/tests/with_rendering/controllers/supervisor_export_image/.gitignore b/tests/api/controllers/supervisor_export_image/.gitignore similarity index 100% rename from tests/with_rendering/controllers/supervisor_export_image/.gitignore rename to tests/api/controllers/supervisor_export_image/.gitignore diff --git a/tests/with_rendering/controllers/supervisor_export_image/Makefile b/tests/api/controllers/supervisor_export_image/Makefile similarity index 100% rename from tests/with_rendering/controllers/supervisor_export_image/Makefile rename to tests/api/controllers/supervisor_export_image/Makefile diff --git a/tests/with_rendering/controllers/supervisor_export_image/supervisor_export_image.c b/tests/api/controllers/supervisor_export_image/supervisor_export_image.c similarity index 100% rename from tests/with_rendering/controllers/supervisor_export_image/supervisor_export_image.c rename to tests/api/controllers/supervisor_export_image/supervisor_export_image.c diff --git a/tests/with_rendering/worlds/billboard.wbt b/tests/api/worlds/billboard.wbt similarity index 100% rename from tests/with_rendering/worlds/billboard.wbt rename to tests/api/worlds/billboard.wbt diff --git a/tests/with_rendering/worlds/supervisor_export_image.wbt b/tests/api/worlds/supervisor_export_image.wbt similarity index 100% rename from tests/with_rendering/worlds/supervisor_export_image.wbt rename to tests/api/worlds/supervisor_export_image.wbt diff --git a/tests/test_suite.py b/tests/test_suite.py index 42ba6cc73aa..582466ff666 100755 --- a/tests/test_suite.py +++ b/tests/test_suite.py @@ -50,7 +50,7 @@ parser.add_argument('--nomake', dest='nomake', default=False, action='store_true', help='The controllers are not re-compiled.') parser.add_argument('--no-ansi-escape', dest='ansi_escape', default=True, action='store_false', help='Disables ansi escape.') parser.add_argument('--group', '-g', type=str, dest='group', default=[], help='Specifies which group of tests should be run.', - choices=['api', 'cache', 'other_api', 'physics', 'protos', 'parser', 'rendering', 'with_rendering']) + choices=['api', 'cache', 'other_api', 'physics', 'protos', 'parser', 'rendering']) parser.add_argument('--performance-log', '-p', type=str, dest='performance_log', default="", help='The name of the performance log file to use if you want to log performance.') parser.add_argument('worlds', nargs='*', default=[]) @@ -236,11 +236,9 @@ def runGroupTest(groupName, firstSimulation, worldsCount, failures): # command.run(silent = False) webotsArguments = [webotsFullPath, firstSimulation] - webotsArguments += ['--mode=fast', '--stdout', '--stderr', '--batch'] + webotsArguments += ['--mode=fast', '--stdout', '--stderr', '--batch', '--no-rendering', '--minimize'] if args.performance_log: webotsArguments += [f'--log-performance={args.performance_log}'] - if groupName != 'with_rendering': - webotsArguments += ['--no-rendering', '--minimize'] if groupName == 'cache': webotsArguments += ['--clear-cache'] command = Command(webotsArguments) @@ -332,7 +330,7 @@ def runGroupTest(groupName, firstSimulation, worldsCount, failures): if args.group: testGroups = [str(args.group)] else: - testGroups = ['api', 'cache', 'other_api', 'physics', 'protos', 'parser', 'rendering', 'with_rendering'] + testGroups = ['api', 'cache', 'other_api', 'physics', 'protos', 'parser', 'rendering'] if sys.platform == 'win32' and 'parser' in testGroups: testGroups.remove('parser') # this one doesn't work on Windows diff --git a/tests/with_rendering/.gitignore b/tests/with_rendering/.gitignore deleted file mode 100644 index 2ced1307bbc..00000000000 --- a/tests/with_rendering/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -/worlds_index.txt -/worlds.txt diff --git a/tests/with_rendering/Makefile b/tests/with_rendering/Makefile deleted file mode 100644 index d9b5df50fe7..00000000000 --- a/tests/with_rendering/Makefile +++ /dev/null @@ -1,16 +0,0 @@ -# Copyright 1996-2023 Cyberbotics Ltd. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -release debug profile clean: - +@make -s -C controllers $(MAKECMDGOALS) diff --git a/tests/with_rendering/controllers/.gitignore b/tests/with_rendering/controllers/.gitignore deleted file mode 100644 index caa17cad8e7..00000000000 --- a/tests/with_rendering/controllers/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -/test_suite_supervisor - diff --git a/tests/with_rendering/controllers/Makefile b/tests/with_rendering/controllers/Makefile deleted file mode 100644 index 3149288896d..00000000000 --- a/tests/with_rendering/controllers/Makefile +++ /dev/null @@ -1,24 +0,0 @@ -# Copyright 1996-2023 Cyberbotics Ltd. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -DIRECTORIES=$(subst test_suite_supervisor,,$(subst Makefile,,$(shell ls))) -TARGETS = $(DIRECTORIES:=.Makefile) - -.PHONY: release debug profile clean - -release debug profile clean: $(TARGETS) - -%.Makefile: - +@echo "# make" $(MAKECMDGOALS) $(@:.Makefile=) - +@make -s -C $(@:.Makefile=) $(MAKECMDGOALS) diff --git a/tests/with_rendering/worlds/.gitignore b/tests/with_rendering/worlds/.gitignore deleted file mode 100644 index 267c479acec..00000000000 --- a/tests/with_rendering/worlds/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -/.*.wbproj -!/.supervisor_export_image.wbproj diff --git a/tests/with_rendering/worlds/.supervisor_export_image.wbproj b/tests/with_rendering/worlds/.supervisor_export_image.wbproj deleted file mode 100644 index e078696581c..00000000000 --- a/tests/with_rendering/worlds/.supervisor_export_image.wbproj +++ /dev/null @@ -1,12 +0,0 @@ -Webots Project File version R2024a -perspectives: 000000ff00000000fd000000020000000100000393000002eafc0200000001fb0000001400540065007800740045006400690074006f00720100000017000002ea0000008900ffffff0000000300000780000000d9fc0100000001fb0000001a0043006f006e0073006f006c00650041006c006c0041006c006c0100000000000007800000006900ffffff000003eb000002ea00000001000000020000000100000008fc00000000 -simulationViewPerspectives: 000000ff000000010000000200000100000005620100000002010000000100 -sceneTreePerspectives: 000000ff00000001000000030000001d000000c0000000fa0100000002010000000200 -maximizedDockId: -1 -centralWidgetVisible: 1 -projectionMode: PERSPECTIVE -userInteractions: selectionDisabled -orthographicViewHeight: 1.75385 -textFiles: 0 "controllers/supervisor_export_image/supervisor_export_image.c" -consoles: Console:All:All -renderingDevicePerspectives: robot:camera;0;1;0;0 From f52605822e0c8099698613c4182edb074dfce2e6 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Mon, 19 Aug 2024 14:29:20 -0700 Subject: [PATCH 04/19] Bump the Qt version from 6.4.3 to 6.5.3. (#6623) * Bump the Qt version from 6.4.3 to 6.5.3. 6.4.3 is no longer supported, while 6.5.3 is the latest LTS release and is supported through 31 Mar 2026. * Add changelog entry. * Get Qt via aqtinstall on Ubuntu and MacOS * Copy system openssl3 libs on Ubuntu 22.04 and use built libs on Ubuntu 20.04. * Build openssl3 from source on Ubuntu 20.04. * Make sure the Contents/Frameworks folder is created. * Make openssl a compilation dependency for Ubuntu 22.04 because we bundle the libs with webots. * Add libxcb-cursor.so.0 (required by Qt intsalled by aqt) to linux packages. --- dependencies/Makefile.linux | 42 ++--- dependencies/Makefile.mac | 7 +- docs/reference/changelog-r2024.md | 1 + .../install/linux_compilation_dependencies.sh | 6 +- scripts/install/linux_runtime_dependencies.sh | 2 +- scripts/install/qt_linux_installer.sh | 151 +++++++----------- scripts/install/qt_mac_installer.sh | 5 +- scripts/packaging/linux_distro.py | 19 +-- 8 files changed, 104 insertions(+), 129 deletions(-) diff --git a/dependencies/Makefile.linux b/dependencies/Makefile.linux index 1321cc71f09..27c0f66dad0 100644 --- a/dependencies/Makefile.linux +++ b/dependencies/Makefile.linux @@ -6,19 +6,20 @@ include $(WEBOTS_HOME)/resources/Makefile.os.include DEPENDENCIES_URL = https://cyberbotics.com/files/repository/dependencies/linux64/release -QT_VERSION = 6.4.3 +QT_VERSION = 6.5.3 QT_PACKAGE = webots-qt-$(QT_VERSION)-linux64-release.tar.bz2 OPENAL_PACKAGE = openal-linux64-1.16.0.tar.bz2 OIS_PACKAGE = libOIS.1.4.tar.bz2 ASSIMP_PACKAGE = libassimp-5.2.3.tar.bz2 PICO_PACKAGE = libpico.tar.bz2 LUA_PACKAGE = lua-5.2.3.tar.gz -OPENSSL_PACKAGE = libssl_1.1.tar.xz +OPENSSL_VERSION=3.0.14 +OPENSSL_SRC_PACKAGE=openssl-$(OPENSSL_VERSION).tar.gz PACKAGES = qt open-al ois pico lua assimp PACKAGES_CLEAN = $(addsuffix -clean, $(PACKAGES)) -ifeq ($(UBUNTU_VERSION), 22.04) +ifeq ($(UBUNTU_VERSION), 20.04) PACKAGES += open-ssl endif @@ -51,9 +52,8 @@ $(WEBOTS_HOME_LIB)/libQt6Core.so.$(QT_VERSION): $(WEBOTS_DEPENDENCY_PATH)/$(QT_P $(WEBOTS_DEPENDENCY_PATH)/$(QT_PACKAGE): @rm -f $(WEBOTS_DEPENDENCY_PATH)/$(QT_PACKAGE) - @echo "# downloading $(QT_PACKAGE)" - @wget -qq $(DEPENDENCIES_URL)/$(QT_PACKAGE) -P $(WEBOTS_DEPENDENCY_PATH) - @if [ "$$(md5sum $(QT_PACKAGE) | awk '{print $$1;}')" != "a5448e2005fa981278f5348b43809932" ]; then echo "MD5 checksum failed for $(QT_PACKAGE)"; exit 1; fi + @echo "# Installing $(QT_PACKAGE)" + $(WEBOTS_HOME)/scripts/install/qt_linux_installer.sh @touch $(WEBOTS_DEPENDENCY_PATH)/$(QT_PACKAGE) @@ -142,20 +142,22 @@ $(WEBOTS_DEPENDENCY_PATH)/lua-5.2.3/src/liblua.a: $(WEBOTS_DEPENDENCY_PATH)/lua- +@make --silent -C $(WEBOTS_DEPENDENCY_PATH)/lua-5.2.3 linux 2> /dev/null open-ssl-clean: - @rm -rf $(WEBOTS_DEPENDENCY_PATH)/openssl-1.1 $(WEBOTS_DEPENDENCY_PATH)/$(OPENSSL_PACKAGE) $(WEBOTS_HOME_LIB)/libcrypto.so* $(WEBOTS_HOME_LIB)/libssl.so* + @rm -rf $(WEBOTS_DEPENDENCY_PATH)/$(OPENSSL_SRC_PACKAGE) $(WEBOTS_DEPENDENCY_PATH)/openssl-$(OPENSSL_VERSION) $(WEBOTS_DEPENDENCY_PATH)/openssl-3.0 $(WEBOTS_HOME_LIB)/libcrypto.so* $(WEBOTS_HOME_LIB)/libssl.so* open-ssl: $(WEBOTS_HOME_LIB)/libcrypto.so -$(WEBOTS_HOME_LIB)/libcrypto.so: $(WEBOTS_DEPENDENCY_PATH)/openssl-1.1 - @cp -a $(WEBOTS_DEPENDENCY_PATH)/openssl-1.1/libcrypto.so* $(WEBOTS_HOME_LIB) - @cp -a $(WEBOTS_DEPENDENCY_PATH)/openssl-1.1/libssl.so* $(WEBOTS_HOME_LIB) - -$(WEBOTS_DEPENDENCY_PATH)/openssl-1.1: - @echo "# downloading $(OPENSSL_PACKAGE)" - @rm -f $(WEBOTS_DEPENDENCY_PATH)/$(OPENSSL_PACKAGE) - @wget -qq $(DEPENDENCIES_URL)/$(OPENSSL_PACKAGE) -P $(WEBOTS_DEPENDENCY_PATH) - @if [ "$$(md5sum $(OPENSSL_PACKAGE) | awk '{print $$1;}')" != "f86af67eb474f562c14c16cfac100aa3" ]; then echo "MD5 checksum failed for $(OPENSSL_PACKAGE)"; exit 1; fi - @echo "# uncompressing $(OPENSSL_PACKAGE)" - @tar xfm $(WEBOTS_DEPENDENCY_PATH)/$(OPENSSL_PACKAGE) -C $(WEBOTS_DEPENDENCY_PATH) - @rm -f $(WEBOTS_DEPENDENCY_PATH)/$(OPENSSL_PACKAGE) - @touch $(WEBOTS_DEPENDENCY_PATH)/$(OPENSSL_PACKAGE) +$(WEBOTS_HOME_LIB)/libcrypto.so: $(WEBOTS_DEPENDENCY_PATH)/openssl-3.0 + @cp -a $(WEBOTS_DEPENDENCY_PATH)/openssl-3.0/libcrypto.so* $(WEBOTS_HOME_LIB) + @cp -a $(WEBOTS_DEPENDENCY_PATH)/openssl-3.0/libssl.so* $(WEBOTS_HOME_LIB) + +$(WEBOTS_DEPENDENCY_PATH)/openssl-3.0: + @echo "# downloading $(OPENSSL_SRC_PACKAGE)" + @rm -f $(WEBOTS_DEPENDENCY_PATH)/$(OPENSSL_SRC_PACKAGE) + @wget -qq https://github.com/openssl/openssl/releases/download/openssl-$(OPENSSL_VERSION)/$(OPENSSL_SRC_PACKAGE) -P $(WEBOTS_DEPENDENCY_PATH) + @if [ "$$(sha256sum $(OPENSSL_SRC_PACKAGE) | awk '{print $$1;}')" != "eeca035d4dd4e84fc25846d952da6297484afa0650a6f84c682e39df3a4123ca" ]; then echo "SHA256 checksum failed for $(OPENSSL_SRC_PACKAGE)"; exit 1; fi + @echo "# uncompressing $(OPENSSL_SRC_PACKAGE)" + @tar xfm $(WEBOTS_DEPENDENCY_PATH)/$(OPENSSL_SRC_PACKAGE) -C $(WEBOTS_DEPENDENCY_PATH) + @cd $(WEBOTS_DEPENDENCY_PATH)/openssl-$(OPENSSL_VERSION) && ./Configure shared --prefix=/usr --openssldir=/usr/lib/ssl --libdir=lib/x86_64-linux-gnu no-idea no-mdc2 no-rc5 no-zlib no-ssl3 enable-unit-test no-ssl3-method enable-rfc3779 enable-cms no-capieng enable-ec_nistp_64_gcc_128 && $(MAKE) -j4 + @mkdir $(WEBOTS_DEPENDENCY_PATH)/openssl-3.0 + @cp -a $(WEBOTS_DEPENDENCY_PATH)/openssl-$(OPENSSL_VERSION)/libcrypto.so* $(WEBOTS_DEPENDENCY_PATH)/openssl-3.0 + @cp -a $(WEBOTS_DEPENDENCY_PATH)/openssl-$(OPENSSL_VERSION)/libssl.so* $(WEBOTS_DEPENDENCY_PATH)/openssl-3.0 diff --git a/dependencies/Makefile.mac b/dependencies/Makefile.mac index 89cff6d0099..cfc31079111 100644 --- a/dependencies/Makefile.mac +++ b/dependencies/Makefile.mac @@ -6,7 +6,7 @@ WEBOTS_HOME_LIB = $(WEBOTS_HOME)/Contents/lib/webots DEPENDENCIES_URL = https://cyberbotics.com/files/repository/dependencies/mac/release WGET = LANG=en_US.UTF-8 wget -qq -QT_PACKAGE = qt-6.4.3-release.tar.bz2 +QT_PACKAGE = qt-6.5.3-release.tar.bz2 ASSIMP_PACKAGE = assimp-5.2.3.tar.bz2 FFMPEG_PACKAGE = ffmpeg.tar.bz2 FREETYPE_PACKAGE = freetype2.tar.bz2 @@ -69,10 +69,9 @@ $(WEBOTS_HOME)/Contents/Frameworks/QtCore.framework/Versions/A/QtCore: $(WEBOTS_ @python3 codesign.py $(WEBOTS_DEPENDENCY_PATH)/$(QT_PACKAGE): - @echo "# downloading $(QT_PACKAGE)" @rm -f $(WEBOTS_DEPENDENCY_PATH)/$(QT_PACKAGE) - @$(WGET) $(DEPENDENCIES_URL)/$(QT_PACKAGE) -P $(WEBOTS_DEPENDENCY_PATH) - @if [ "71c339a01c8cf324bc8c00b39d6fbd1e" != `md5 -q $(QT_PACKAGE)` ]; then echo "MD5 checksum failed for $(QT_PACKAGE)"; exit 1; fi + @echo "# Installing $(QT_PACKAGE)" + $(WEBOTS_HOME)/scripts/install/qt_mac_installer.sh @touch $(WEBOTS_DEPENDENCY_PATH)/$(QT_PACKAGE) assimp-clean: diff --git a/docs/reference/changelog-r2024.md b/docs/reference/changelog-r2024.md index 91cf19d8809..73ca28e3a52 100644 --- a/docs/reference/changelog-r2024.md +++ b/docs/reference/changelog-r2024.md @@ -23,3 +23,4 @@ Released on December **th, 2023. - Fixed incomplete loading while minimized under some windowing systems ([#6617](https://github.com/cyberbotics/webots/pull/6617)). - Fixed unitialized sliding friction when using asymmetric rolling friction ([#6618](https://github.com/cyberbotics/webots/pull/6618)). - Made `supervisor_export_image` work even if using the `--minimize --no-rendering` options ([#6622](https://github.com/cyberbotics/webots/pull/6622)). + - Fixed to use a version of Qt that is still supported ([#6623](https://github.com/cyberbotics/webots/pull/6623)). diff --git a/scripts/install/linux_compilation_dependencies.sh b/scripts/install/linux_compilation_dependencies.sh index 59828a50d16..acb2fe6bece 100755 --- a/scripts/install/linux_compilation_dependencies.sh +++ b/scripts/install/linux_compilation_dependencies.sh @@ -12,13 +12,13 @@ fi alias apt='apt --option="APT::Acquire::Retries=3"' apt update -apt install --yes git lsb-release cmake swig libglu1-mesa-dev libglib2.0-dev libfreeimage3 libfreetype6-dev libxml2-dev libboost-dev libssh-gcrypt-dev libzip-dev libreadline-dev pbzip2 wget zip unzip +apt install --yes git lsb-release cmake swig libglu1-mesa-dev libglib2.0-dev libfreeimage3 libfreetype6-dev libxml2-dev libboost-dev libssh-gcrypt-dev libzip-dev libreadline-dev pbzip2 wget zip unzip python3 python3-pip UBUNTU_VERSION=$(lsb_release -rs) if [[ $UBUNTU_VERSION == "20.04" ]]; then - apt install --yes libzip5 + apt install --yes libzip5 perl libtext-template-perl elif [[ $UBUNTU_VERSION == "22.04" ]]; then - apt install --yes libzip4 + apt install --yes libzip4 openssl else echo "Unsupported Linux version: dependencies may not be completely installed. Only the two latest Ubuntu LTS are supported." fi diff --git a/scripts/install/linux_runtime_dependencies.sh b/scripts/install/linux_runtime_dependencies.sh index 92035629d66..c2011cd3cdc 100755 --- a/scripts/install/linux_runtime_dependencies.sh +++ b/scripts/install/linux_runtime_dependencies.sh @@ -12,7 +12,7 @@ fi alias apt='apt --option="APT::Acquire::Retries=3"' apt update -apt install --yes lsb-release g++ make libavcodec-extra libglu1-mesa libegl1 libxkbcommon-x11-dev libxcb-keysyms1 libxcb-image0 libxcb-icccm4 libxcb-randr0 libxcb-render-util0 libxcb-xinerama0 libxcomposite-dev libxtst6 libnss3 +apt install --yes lsb-release g++ make libavcodec-extra libglu1-mesa libegl1 libxkbcommon-x11-dev libxcb-keysyms1 libxcb-image0 libxcb-icccm4 libxcb-randr0 libxcb-render-util0 libxcb-xinerama0 libxcomposite-dev libxtst6 libnss3 libxcb-cursor0 if [[ -z "$DISPLAY" ]]; then apt install --yes xvfb fi diff --git a/scripts/install/qt_linux_installer.sh b/scripts/install/qt_linux_installer.sh index d44cfa4402c..22636f4f38c 100755 --- a/scripts/install/qt_linux_installer.sh +++ b/scripts/install/qt_linux_installer.sh @@ -3,11 +3,17 @@ # exit when any command fails set -e -# follow the instructions from https://github.com/cyberbotics/webots/wiki/Qt-compilation#linux to download and compile Qt before executing this script. - -QT_VERSION=6.4.3 -ICU_VERSION=56 +QT_VERSION=6.5.3 +pip install --no-input aqtinstall +aqt install-qt --outputdir ~/Qt linux desktop ${QT_VERSION} gcc_64 -m qtwebsockets QT_INSTALLATION_PATH=~/Qt/${QT_VERSION}/gcc_64 +QT_INSTALLATION_BIN_PATH=${QT_INSTALLATION_PATH}/bin +QT_INSTALLATION_LIBEXEC_PATH=${QT_INSTALLATION_PATH}/libexec +QT_INSTALLATION_LIB_PATH=${QT_INSTALLATION_PATH}/lib +QT_INSTALLATION_INCLUDE_PATH=${QT_INSTALLATION_PATH}/include +QT_INSTALLATION_PLUGINS_PATH=${QT_INSTALLATION_PATH}/plugins +QT_INSTALLATION_TRANSLATIONS_PATH=${QT_INSTALLATION_PATH}/translations + WEBOTS_HOME="$(cd "$(dirname "${BASH_SOURCE[0]}" )"/../.. && pwd)" echo Installing Qt in Webots @@ -55,96 +61,59 @@ mkdir lib/webots/qt/plugins/wayland-decoration-client mkdir lib/webots/qt/resources mkdir lib/webots/qt/translations -cp $QT_INSTALLATION_PATH/bin/lrelease bin/qt/ -cp $QT_INSTALLATION_PATH/bin/lupdate bin/qt/ -cp $QT_INSTALLATION_PATH/libexec/moc bin/qt/ -cp -r $QT_INSTALLATION_PATH/include/QtConcurrent include/qt/QtConcurrent/ -cp -r $QT_INSTALLATION_PATH/include/QtCore include/qt/QtCore/ -cp -r $QT_INSTALLATION_PATH/include/QtGui include/qt/QtGui/ -cp -r $QT_INSTALLATION_PATH/include/QtNetwork include/qt/QtNetwork/ -cp -r $QT_INSTALLATION_PATH/include/QtOpenGL include/qt/QtOpenGL/ -cp -r $QT_INSTALLATION_PATH/include/QtOpenGLWidgets include/qt/QtOpenGLWidgets/ -cp -r $QT_INSTALLATION_PATH/include/QtPrintSupport include/qt/QtPrintSupport/ -cp -r $QT_INSTALLATION_PATH/include/QtQml include/qt/QtQml/ -cp -r $QT_INSTALLATION_PATH/include/QtWebSockets include/qt/QtWebSockets/ -cp -r $QT_INSTALLATION_PATH/include/QtWidgets include/qt/QtWidgets/ -cp -r $QT_INSTALLATION_PATH/include/QtXml include/qt/QtXml/ +cp $QT_INSTALLATION_BIN_PATH/lrelease bin/qt/ +cp $QT_INSTALLATION_BIN_PATH/lupdate bin/qt/ +cp $QT_INSTALLATION_LIBEXEC_PATH/moc bin/qt/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtConcurrent include/qt/QtConcurrent/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtCore include/qt/QtCore/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtGui include/qt/QtGui/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtNetwork include/qt/QtNetwork/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtOpenGL include/qt/QtOpenGL/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtOpenGLWidgets include/qt/QtOpenGLWidgets/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtPrintSupport include/qt/QtPrintSupport/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtQml include/qt/QtQml/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtWebSockets include/qt/QtWebSockets/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtWidgets include/qt/QtWidgets/ +cp -r $QT_INSTALLATION_INCLUDE_PATH/QtXml include/qt/QtXml/ rm -rf include/qt/Qt*/*/$QT_VERSION -cp $QT_INSTALLATION_PATH/lib/libQt6Concurrent.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6Core.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6DBus.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6Gui.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6Network.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6OpenGL.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6OpenGLWidgets.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6PrintSupport.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6Qml.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6WebSockets.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6Widgets.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6XcbQpa.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libQt6Xml.so.$QT_VERSION lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libicudata.so.$ICU_VERSION.1 lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libicui18n.so.$ICU_VERSION.1 lib/webots/ -cp $QT_INSTALLATION_PATH/lib/libicuuc.so.$ICU_VERSION.1 lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6Concurrent.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6Core.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6DBus.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6Gui.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6Network.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6OpenGL.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6OpenGLWidgets.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6PrintSupport.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6Qml.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6WaylandClient.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6WaylandEglClientHwIntegration.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6WebSockets.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6Widgets.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6XcbQpa.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libQt6Xml.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libicudata.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libicui18n.so* lib/webots/ +cp -a $QT_INSTALLATION_LIB_PATH/libicuuc.so* lib/webots/ echo $'[Paths]\nPrefix = ..\n' > lib/webots/qt/libexec/qt.conf -cp $QT_INSTALLATION_PATH/plugins/platforms/libqxcb.so lib/webots/qt/plugins/platforms/ -cp $QT_INSTALLATION_PATH/plugins/platforms/libqwayland-egl.so lib/webots/qt/plugins/platforms/ -cp $QT_INSTALLATION_PATH/plugins/platforms/libqwayland-generic.so lib/webots/qt/plugins/platforms/ -cp $QT_INSTALLATION_PATH/plugins/platformthemes/libqgtk3.so lib/webots/qt/plugins/platformthemes/ -cp $QT_INSTALLATION_PATH/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so lib/webots/qt/plugins/platforminputcontexts/ -cp $QT_INSTALLATION_PATH/plugins/platforminputcontexts/libibusplatforminputcontextplugin.so lib/webots/qt/plugins/platforminputcontexts/ -cp $QT_INSTALLATION_PATH/plugins/printsupport/libcupsprintersupport.so lib/webots/qt/plugins/printsupport/ -cp $QT_INSTALLATION_PATH/plugins/tls/*.so lib/webots/qt/plugins/tls/ -cp $QT_INSTALLATION_PATH/plugins/xcbglintegrations/libqxcb-glx-integration.so lib/webots/qt/plugins/xcbglintegrations/ -cp $QT_INSTALLATION_PATH/plugins/wayland-graphics-integration-client/* lib/webots/qt/plugins/wayland-graphics-integration-client/ -cp $QT_INSTALLATION_PATH/plugins/wayland-shell-integration/* lib/webots/qt/plugins/wayland-shell-integration/ -cp $QT_INSTALLATION_PATH/plugins/wayland-decoration-client/* lib/webots/qt/plugins/wayland-decoration-client/ -cp $QT_INSTALLATION_PATH/plugins/imageformats/libqjpeg.so lib/webots/qt/plugins/imageformats/ -cp -r $QT_INSTALLATION_PATH/translations/qt_* lib/webots/qt/translations/ -cp -r $QT_INSTALLATION_PATH/translations/qtbase_* lib/webots/qt/translations/ -cp -r $QT_INSTALLATION_PATH/translations/qtdeclarative_* lib/webots/qt/translations/ -cp -r $QT_INSTALLATION_PATH/translations/qtwebsockets_* lib/webots/qt/translations/ - -cd lib/webots -ln -sf libQt6Concurrent.so.$QT_VERSION libQt6Concurrent.so.6 -ln -sf libQt6Concurrent.so.$QT_VERSION libQt6Concurrent.so -ln -sf libQt6Core.so.$QT_VERSION libQt6Core.so.6 -ln -sf libQt6Core.so.$QT_VERSION libQt6Core.so -ln -sf libQt6DBus.so.$QT_VERSION libQt6DBus.so.6 -ln -sf libQt6DBus.so.$QT_VERSION libQt6DBus.so -ln -sf libQt6Gui.so.$QT_VERSION libQt6Gui.so.6 -ln -sf libQt6Gui.so.$QT_VERSION libQt6Gui.so -ln -sf libQt6Network.so.$QT_VERSION libQt6Network.so.6 -ln -sf libQt6Network.so.$QT_VERSION libQt6Network.so -ln -sf libQt6OpenGL.so.$QT_VERSION libQt6OpenGL.so.6 -ln -sf libQt6OpenGL.so.$QT_VERSION libQt6OpenGL.so -ln -sf libQt6OpenGLWidgets.so.$QT_VERSION libQt6OpenGLWidgets.so.6 -ln -sf libQt6OpenGLWidgets.so.$QT_VERSION libQt6OpenGLWidgets.so -ln -sf libQt6PrintSupport.so.$QT_VERSION libQt6PrintSupport.so.6 -ln -sf libQt6PrintSupport.so.$QT_VERSION libQt6PrintSupport.so -ln -sf libQt6Qml.so.$QT_VERSION libQt6Qml.so.6 -ln -sf libQt6Qml.so.$QT_VERSION libQt6Qml.so -ln -sf libQt6WebSockets.so.$QT_VERSION libQt6WebSockets.so.6 -ln -sf libQt6WebSockets.so.$QT_VERSION libQt6WebSockets.so -ln -sf libQt6Widgets.so.$QT_VERSION libQt6Widgets.so.6 -ln -sf libQt6Widgets.so.$QT_VERSION libQt6Widgets.so -ln -sf libQt6Xml.so.$QT_VERSION libQt6Xml.so.6 -ln -sf libQt6Xml.so.$QT_VERSION libQt6Xml.so -ln -sf libQt6XcbQpa.so.$QT_VERSION libQt6XcbQpa.so.6 -ln -sf libQt6XcbQpa.so.$QT_VERSION libQt6XcbQpa.so -ln -sf libQt6WaylandClient.so.$QT_VERSION libQt6WaylandClient.so.6 -ln -sf libQt6WaylandClient.so.$QT_VERSION libQt6WaylandClient.so -ln -sf libQt6WaylandEglClientHwIntegration.so.$QT_VERSION libQt6WaylandEglClientHwIntegration.so.6 -ln -sf libQt6WaylandEglClientHwIntegration.so.$QT_VERSION libQt6WaylandEglClientHwIntegration.so -ln -sf libQt6WlShellIntegration.so.$QT_VERSION libQt6WlShellIntegration.so.6 -ln -sf libQt6WlShellIntegration.so.$QT_VERSIOn libQt6WlShellIntegration.so -ln -sf libicudata.so.$ICU_VERSION.1 libicudata.so.$ICU_VERSION -ln -sf libicui18n.so.$ICU_VERSION.1 libicui18n.so.$ICU_VERSION -ln -sf libicuuc.so.$ICU_VERSION.1 libicuuc.so.$ICU_VERSION - -cd ../.. +cp -a $QT_INSTALLATION_PLUGINS_PATH/platforms/libqxcb.so lib/webots/qt/plugins/platforms/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/platforms/libqwayland-egl.so lib/webots/qt/plugins/platforms/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/platforms/libqwayland-generic.so lib/webots/qt/plugins/platforms/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/platformthemes/* lib/webots/qt/plugins/platformthemes/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/platforminputcontexts/libcomposeplatforminputcontextplugin.so lib/webots/qt/plugins/platforminputcontexts/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/platforminputcontexts/libibusplatforminputcontextplugin.so lib/webots/qt/plugins/platforminputcontexts/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/printsupport/libcupsprintersupport.so lib/webots/qt/plugins/printsupport/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/tls/*.so lib/webots/qt/plugins/tls/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/xcbglintegrations/libqxcb-glx-integration.so lib/webots/qt/plugins/xcbglintegrations/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/wayland-graphics-integration-client/* lib/webots/qt/plugins/wayland-graphics-integration-client/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/wayland-shell-integration/* lib/webots/qt/plugins/wayland-shell-integration/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/wayland-decoration-client/* lib/webots/qt/plugins/wayland-decoration-client/ +cp -a $QT_INSTALLATION_PLUGINS_PATH/imageformats/libqjpeg.so lib/webots/qt/plugins/imageformats/ +cp -a $QT_INSTALLATION_TRANSLATIONS_PATH/qt_* lib/webots/qt/translations/ +cp -a $QT_INSTALLATION_TRANSLATIONS_PATH/qtbase_* lib/webots/qt/translations/ +cp -a $QT_INSTALLATION_TRANSLATIONS_PATH/qtdeclarative_* lib/webots/qt/translations/ +cp -a $QT_INSTALLATION_TRANSLATIONS_PATH/qtwebsockets_* lib/webots/qt/translations/ -ARCHIVE=webots-qt-$QT_VERSION-linux64-release.tar.bz2 +ARCHIVE=dependencies/webots-qt-$QT_VERSION-linux64-release.tar.bz2 echo Compressing $ARCHIVE \(please wait\) tar cjf $ARCHIVE lib/webots/libQt6* lib/webots/libicu* lib/webots/qt include/qt bin/qt/lrelease bin/qt/lupdate bin/qt/moc diff --git a/scripts/install/qt_mac_installer.sh b/scripts/install/qt_mac_installer.sh index 5c70a02fb43..35a6d84829b 100755 --- a/scripts/install/qt_mac_installer.sh +++ b/scripts/install/qt_mac_installer.sh @@ -8,7 +8,9 @@ if [[ -z "${WEBOTS_HOME}" ]]; then exit 1 fi -QT_VERSION=6.4.3 +QT_VERSION=6.5.3 +pip install --no-input aqtinstall +aqt install-qt --outputdir $HOME/Qt mac desktop ${QT_VERSION} clang_64 -m qtwebsockets # prepare Webots cd $WEBOTS_HOME @@ -37,6 +39,7 @@ declare -a qtFrameworks=( \ "QtXml" \ ) +mkdir -p $WEBOTS_HOME/Contents/Frameworks for f in "${qtFrameworks[@]}" do cp -R lib/$f.framework $WEBOTS_HOME/Contents/Frameworks diff --git a/scripts/packaging/linux_distro.py b/scripts/packaging/linux_distro.py index a94dc61a87c..f19916cf728 100644 --- a/scripts/packaging/linux_distro.py +++ b/scripts/packaging/linux_distro.py @@ -44,7 +44,8 @@ class LinuxWebotsPackage(WebotsPackage): "libxcb-icccm.so.4", "libxcb-randr.so.0", "libxcb-render-util.so.0", - "libxcb-xinerama.so.0" + "libxcb-xinerama.so.0", + "libxcb-cursor.so.0" ] USR_LIB_X68_64_20_04 = [ "libHalf.so.24", @@ -122,17 +123,17 @@ def create_webots_bundle(self, include_commit_file): # copy OpenSSL libraries from Ubuntu 20.04 system and needed on Ubuntu 22.04 system_lib_path = os.path.join('/usr', 'lib', 'x86_64-linux-gnu') package_webots_lib = os.path.join(self.package_webots_path, 'lib', 'webots') - if distro.version() == '22.04': - shutil.copy(os.path.join(self.webots_home, 'lib', 'webots', 'libcrypto.so.1.1'), - os.path.join(package_webots_lib, 'libcrypto.so.1.1')) + if distro.version() == '20.04': + shutil.copy(os.path.join(self.webots_home, 'lib', 'webots', 'libcrypto.so.3'), + os.path.join(package_webots_lib, 'libcrypto.so.3')) shutil.copy(os.path.join(self.webots_home, 'lib', 'webots', 'libcrypto.so'), os.path.join(package_webots_lib, 'libcrypto.so')) - shutil.copy(os.path.join(self.webots_home, 'lib', 'webots', 'libssl.so.1.1'), - os.path.join(package_webots_lib, 'libssl.so.1.1')) + shutil.copy(os.path.join(self.webots_home, 'lib', 'webots', 'libssl.so.3'), + os.path.join(package_webots_lib, 'libssl.so.3')) shutil.copy(os.path.join(self.webots_home, 'lib', 'webots', 'libssl.so'), os.path.join(package_webots_lib, 'libssl.so')) - else: # Ubuntu 20.04 - openssl_libs = ['libcrypto.so.1.1', 'libcrypto.so', 'libssl.so.1.1', 'libssl.so'] + else: # Ubuntu 22.04 + openssl_libs = ['libcrypto.so.3', 'libcrypto.so', 'libssl.so.3', 'libssl.so'] for lib in openssl_libs: shutil.copy(os.path.join(system_lib_path, lib), package_webots_lib) @@ -194,7 +195,7 @@ def create_debian_bundle(self): "libnss3, libstdc++6 (>= 4.0.2-4), libxaw7, libxrandr2, libxrender1, " "libssh-dev, libzip-dev, xserver-xorg-core, libxslt1.1, " "libfreetype6, libxkbcommon-x11-0, libxcb-keysyms1, libxcb-image0, libxcb-icccm4, " - "libxcb-randr0, libxcb-render-util0, libxcb-xinerama0\n" + "libxcb-randr0, libxcb-render-util0, libxcb-xinerama0, libxcb-cursor0\n" "Conflicts: webots-for-nao\n" "Maintainer: Olivier Michel \n" "Description: Mobile robot simulation software\n" From 6ab65d257b1e12c42576b8dc160df6a1251d2007 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Mon, 19 Aug 2024 15:20:45 -0700 Subject: [PATCH 05/19] Remove and ignore epuck.urdf file created by test. (#6629) Looks like this was accidentally committed a couple years ago. --- tests/api/controllers/robot_urdf/.gitignore | 1 + tests/api/controllers/robot_urdf/epuck.urdf | 421 -------------------- 2 files changed, 1 insertion(+), 421 deletions(-) delete mode 100644 tests/api/controllers/robot_urdf/epuck.urdf diff --git a/tests/api/controllers/robot_urdf/.gitignore b/tests/api/controllers/robot_urdf/.gitignore index eb4ff860d35..5117d40a815 100644 --- a/tests/api/controllers/robot_urdf/.gitignore +++ b/tests/api/controllers/robot_urdf/.gitignore @@ -3,3 +3,4 @@ /tiago.urdf /ur5e.urdf /prob3.urdf +/epuck.urdf diff --git a/tests/api/controllers/robot_urdf/epuck.urdf b/tests/api/controllers/robot_urdf/epuck.urdf deleted file mode 100644 index 1b56f66eb9f..00000000000 --- a/tests/api/controllers/robot_urdf/epuck.urdf +++ /dev/null @@ -1,421 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - From 4da37fe0fdb1438dbc3008b7fe7d937482164a3a Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Mon, 19 Aug 2024 21:55:01 -0700 Subject: [PATCH 06/19] Add msys2 package that includes CA certificates. (#6628) * Add msys2 package that includes CA certificates. These are needed to run test_suite.py. * Add changelog entry. --- docs/reference/changelog-r2024.md | 1 + scripts/install/msys64_installer.sh | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/reference/changelog-r2024.md b/docs/reference/changelog-r2024.md index 73ca28e3a52..60778a9ceb2 100644 --- a/docs/reference/changelog-r2024.md +++ b/docs/reference/changelog-r2024.md @@ -23,4 +23,5 @@ Released on December **th, 2023. - Fixed incomplete loading while minimized under some windowing systems ([#6617](https://github.com/cyberbotics/webots/pull/6617)). - Fixed unitialized sliding friction when using asymmetric rolling friction ([#6618](https://github.com/cyberbotics/webots/pull/6618)). - Made `supervisor_export_image` work even if using the `--minimize --no-rendering` options ([#6622](https://github.com/cyberbotics/webots/pull/6622)). + - Fixed CA certificates needed by `test_suite.py` missing in Windows development environment ([#6628](https://github.com/cyberbotics/webots/pull/6628)). - Fixed to use a version of Qt that is still supported ([#6623](https://github.com/cyberbotics/webots/pull/6623)). diff --git a/scripts/install/msys64_installer.sh b/scripts/install/msys64_installer.sh index c6b8a2ce5bc..4c6cae84c90 100644 --- a/scripts/install/msys64_installer.sh +++ b/scripts/install/msys64_installer.sh @@ -26,6 +26,7 @@ declare -a BASE_PACKAGES=( "mingw-w64-x86_64-ffmpeg" # Webots movies "mingw-w64-x86_64-dlfcn" # dependency of ffmpeg "mingw-w64-x86_64-python" # python controllers and scripts + "mingw-w64-x86_64-ca-certificates" # CA certs needed to run tests/test_suite.py ) declare -a OPTIONAL_PACKAGES=( From 3485871c10f5f5aee8b99b49d27624ba98885380 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Sun, 25 Aug 2024 07:29:48 -0700 Subject: [PATCH 07/19] Mostly MacOS-related fixes (#6635) * Support running qt_mac_installer.sh (and thus make) as regular user. * Use NULL instead of kIOMainPortDefault or kIOMasterPortDefault to support compilation across SDK versions. * Don't reuse socket that fails to connect because doing so causes "Invalid argument" error on MacOS. * Fix long robot names causing connection problems in some environments. * Improve test failure message when check_urdf is not installed. * Ensure that the same format is used for the contexts associated with secondary windows, so that sharing will be successful. * Use venv for aqt on MacOS. * Fix ipc server name encoding to work with extern controllers. * Fix crash on macos when exiting after running check_saved_hidden_parameters.wbt. --- docs/reference/changelog-r2024.md | 3 ++ .../e-puck_bluetooth/Serial.cpp | 3 +- scripts/install/qt_mac_installer.sh | 6 ++- src/controller/c/robot.c | 42 ++++++++++++------- src/webots/gui/WbRenderingDeviceWindow.cpp | 2 +- src/webots/nodes/WbRobot.cpp | 24 ++++++++--- src/webots/wren/WbWrenCamera.cpp | 7 +++- tests/api/controllers/robot_urdf/robot_urdf.c | 4 +- tests/test_suite.py | 22 +++++----- 9 files changed, 77 insertions(+), 36 deletions(-) diff --git a/docs/reference/changelog-r2024.md b/docs/reference/changelog-r2024.md index 60778a9ceb2..e27c7cb6361 100644 --- a/docs/reference/changelog-r2024.md +++ b/docs/reference/changelog-r2024.md @@ -25,3 +25,6 @@ Released on December **th, 2023. - Made `supervisor_export_image` work even if using the `--minimize --no-rendering` options ([#6622](https://github.com/cyberbotics/webots/pull/6622)). - Fixed CA certificates needed by `test_suite.py` missing in Windows development environment ([#6628](https://github.com/cyberbotics/webots/pull/6628)). - Fixed to use a version of Qt that is still supported ([#6623](https://github.com/cyberbotics/webots/pull/6623)). + - Fixed connection errors when using long robot names in some environments ([#6635](https://github.com/cyberbotics/webots/pull/6635)). + - Fixed crash on macos when closing Webots under some circumstances ([#6635](https://github.com/cyberbotics/webots/pull/6635)). + - Fixed error on macos when when putting displays and cameras in separate windows ([#6635](https://github.com/cyberbotics/webots/pull/6635)). diff --git a/projects/robots/gctronic/e-puck/plugins/remote_controls/e-puck_bluetooth/Serial.cpp b/projects/robots/gctronic/e-puck/plugins/remote_controls/e-puck_bluetooth/Serial.cpp index d5f79e32eee..de6412ba8b9 100644 --- a/projects/robots/gctronic/e-puck/plugins/remote_controls/e-puck_bluetooth/Serial.cpp +++ b/projects/robots/gctronic/e-puck/plugins/remote_controls/e-puck_bluetooth/Serial.cpp @@ -384,7 +384,8 @@ void Serial::updatePorts() { printf("IOServiceMatching failed.\n"); CFDictionarySetValue(classesToMatch, CFSTR(kIOSerialBSDTypeKey), CFSTR(kIOSerialBSDRS232Type)); - kernResult = IOServiceGetMatchingServices(kIOMainPortDefault, classesToMatch, &serialPortIterator); + // NULL is equivalent to kIOMainPortDefault or kIOMasterPortDefault but works across SDK versions + kernResult = IOServiceGetMatchingServices((mach_port_t)NULL, classesToMatch, &serialPortIterator); if (kernResult != KERN_SUCCESS) printf("IOServiceGetMatchingServices failed: %d\n", kernResult); io_object_t service; diff --git a/scripts/install/qt_mac_installer.sh b/scripts/install/qt_mac_installer.sh index 35a6d84829b..acfc8f4d6d5 100755 --- a/scripts/install/qt_mac_installer.sh +++ b/scripts/install/qt_mac_installer.sh @@ -9,8 +9,12 @@ if [[ -z "${WEBOTS_HOME}" ]]; then fi QT_VERSION=6.5.3 -pip install --no-input aqtinstall +python3 -m venv ~/python_env_for_aqt +source ~/python_env_for_aqt/bin/activate +python3 -m pip install --upgrade pip +python3 -m pip install --no-input aqtinstall aqt install-qt --outputdir $HOME/Qt mac desktop ${QT_VERSION} clang_64 -m qtwebsockets +rm -rf ~/python_env_for_aqt # prepare Webots cd $WEBOTS_HOME diff --git a/src/controller/c/robot.c b/src/controller/c/robot.c index 8b32ca8c1ef..50616de6680 100644 --- a/src/controller/c/robot.c +++ b/src/controller/c/robot.c @@ -1081,21 +1081,33 @@ static void wb_robot_cleanup_devices() { } } -static char *encode_robot_name(const char *robot_name) { +static char *encode_robot_name(const char *robot_name, int chars_used) { if (!robot_name) return NULL; char *encoded_name = percent_encode(robot_name); int length = strlen(encoded_name); // the robot name is used to connect to the libController and in this process there are indirect - // limitations such as QLocalServer only accepting strings up to 106 characters for server names, - // for these reasons if the robot name is bigger than an arbitrary length, a hashed version is used instead - if (length > 70) { // note: this threshold should be the same as in WbRobot.cpp + // limitations such as QLocalServer only accepting strings up to 91 characters long for server names + // on some platforms. + // Since the server name also contains the tmp path and that includes the user's username and, + // in the case of a Snap, the user's home directory, we need to limit the length of the encoded + // robot name based on the tmp path. If it is longer than that, then we compute a hashed version + // of the name and use as much of it as we can. If that would be less than 4 chars, we try to use + // 4 chars and hope we are on a platform where QLocalServer accepts longer names. 4 chars makes the + // chance of a name collision 1/65536. + // Note: It is critical that the same logic is used in WbRobot.cpp + int max_name_length = 91 - chars_used; + if (max_name_length < 4) + max_name_length = 4; + // Round down to the next multiple of 2 because it makes the code easier. + max_name_length = max_name_length / 2 * 2; + if (length > max_name_length) { char hash[21]; - char *output = malloc(41); + char *output = malloc(max_name_length + 1); SHA1(hash, encoded_name, length); free(encoded_name); - for (size_t i = 0; i < 20; i++) + for (size_t i = 0; i < max_name_length / 2 && i < 20; i++) sprintf((output + (2 * i)), "%02x", hash[i] & 0xff); return output; } @@ -1104,10 +1116,12 @@ static char *encode_robot_name(const char *robot_name) { } static char *compute_socket_filename(char *error_buffer) { - char *robot_name = encode_robot_name(wbu_system_getenv("WEBOTS_ROBOT_NAME")); const char *WEBOTS_INSTANCE_PATH = wbu_system_webots_instance_path(true); + char *robot_name = NULL; + const char *WEBOTS_ROBOT_NAME = wbu_system_getenv("WEBOTS_ROBOT_NAME"); char *socket_filename; - if (robot_name && robot_name[0] && WEBOTS_INSTANCE_PATH && WEBOTS_INSTANCE_PATH[0]) { + if (WEBOTS_ROBOT_NAME && WEBOTS_ROBOT_NAME[0] && WEBOTS_INSTANCE_PATH && WEBOTS_INSTANCE_PATH[0]) { + robot_name = encode_robot_name(WEBOTS_ROBOT_NAME, strlen(WEBOTS_INSTANCE_PATH) + strlen("ipc//intern")); #ifndef _WIN32 const int length = strlen(WEBOTS_INSTANCE_PATH) + strlen(robot_name) + 15; // "%sintern/%s/socket" socket_filename = malloc(length); @@ -1267,7 +1281,7 @@ static char *compute_socket_filename(char *error_buffer) { free(robot_name); const char *sub_string = strstr(&WEBOTS_CONTROLLER_URL[6], "/"); - robot_name = encode_robot_name(sub_string ? sub_string + 1 : NULL); + robot_name = encode_robot_name(sub_string ? sub_string + 1 : NULL, strlen(ipc_folder) + strlen("//extern")); if (robot_name) { #ifndef _WIN32 // socket file name is like: folder + robot_name + "/extern" @@ -1370,7 +1384,7 @@ static void compute_remote_info(char **host, int *port, char **robot_name) { snprintf(*host, host_length, "%s", &WEBOTS_CONTROLLER_URL[6]); sscanf(url_suffix, ":%d", port); const char *rn = strstr(url_suffix, "/"); - *robot_name = rn != NULL ? encode_robot_name(rn + 1) : NULL; + *robot_name = rn != NULL ? encode_robot_name(rn + 1, 0) : NULL; } int wb_robot_init() { // API initialization @@ -1462,11 +1476,11 @@ int wb_robot_init() { // API initialization retry -= 5; fprintf(stderr, "%s, pending until loading is done...\n", error_message); } else if (socket_filename) { + fprintf(stderr, + "The specified robot (at %s) is not in the list of robots with controllers, retrying for another %d " + "seconds...\n", + socket_filename, 50 - retry); free(socket_filename); - fprintf( - stderr, - "The specified robot is not in the list of robots with controllers, retrying for another %d seconds...\n", - 50 - retry); } else fprintf(stderr, "%s, retrying for another %d seconds...\n", error_message, 50 - retry); } diff --git a/src/webots/gui/WbRenderingDeviceWindow.cpp b/src/webots/gui/WbRenderingDeviceWindow.cpp index 282d1731cc1..1efe540efc1 100644 --- a/src/webots/gui/WbRenderingDeviceWindow.cpp +++ b/src/webots/gui/WbRenderingDeviceWindow.cpp @@ -303,7 +303,7 @@ void WbRenderingDeviceWindow::renderNow() { if (!mContext) { mContext = new QOpenGLContext(this); - mContext->setFormat(requestedFormat()); + mContext->setFormat(cMainOpenGLContext->format()); mContext->setShareContext(cMainOpenGLContext); mContext->create(); } diff --git a/src/webots/nodes/WbRobot.cpp b/src/webots/nodes/WbRobot.cpp index 143047aa5f3..821ba012c9a 100644 --- a/src/webots/nodes/WbRobot.cpp +++ b/src/webots/nodes/WbRobot.cpp @@ -59,6 +59,7 @@ #include #include +#include #include static QHash createSpecialKeys() { @@ -1093,12 +1094,25 @@ void WbRobot::dispatchAnswer(WbDataStream &stream, bool includeDevices) { } QString WbRobot::encodedName() const { - const QString encodedName = QUrl::toPercentEncoding(name()); + QString encodedName = QUrl::toPercentEncoding(name()); // the robot name is used to connect to the libController and in this process there are indirect - // limitations such as QLocalServer only accepting strings up to 106 characters for server names, - // for these reasons if the robot name is bigger than an arbitrary length, a hashed version is used instead - if (encodedName.length() > 70) // note: this threshold should be the same as in robot.c - return QString(QCryptographicHash::hash(encodedName.toUtf8(), QCryptographicHash::Sha1).toHex()); + // limitations such as QLocalServer only accepting strings up to 91 characters long for server names + // on some platforms. + // Since the server name also contains the tmp path and that includes the user's username and, + // in the case of a Snap, the user's home directory, we need to limit the length of the encoded + // robot name based on the tmp path. If it is longer than that, then we compute a hashed version + // of the name and use as much of it as we can. If that would be less than 4 chars, we try to use + // 4 chars and hope we are on a platform where QLocalServer accepts longer names. 4 chars makes the + // chance of a name collision 1/65536. + // Note: It is critical that the same logic is used in robot.c + const QString &tmpPath = WbStandardPaths::webotsTmpPath(); + qsizetype maxNameLength = std::max((qsizetype)4, 91 - (tmpPath.length() + (qsizetype)strlen("ipc//extern"))); + // Round down to the next multiple of 2 because it makes the code in robot.c easier. + maxNameLength = maxNameLength / 2 * 2; + if (encodedName.length() > maxNameLength) { + encodedName = QString(QCryptographicHash::hash(encodedName.toUtf8(), QCryptographicHash::Sha1).toHex()); + encodedName.truncate(maxNameLength); + } return encodedName; } diff --git a/src/webots/wren/WbWrenCamera.cpp b/src/webots/wren/WbWrenCamera.cpp index eedf0c47c48..bfafa2580eb 100644 --- a/src/webots/wren/WbWrenCamera.cpp +++ b/src/webots/wren/WbWrenCamera.cpp @@ -664,10 +664,13 @@ void WbWrenCamera::cleanup() { } WrTextureRtt *renderingTexture = wr_frame_buffer_get_output_texture(mResultFrameBuffer, 0); - WrTextureRtt *outputTexture = wr_frame_buffer_get_output_texture(mResultFrameBuffer, 1); - wr_frame_buffer_delete(mResultFrameBuffer); wr_texture_delete(WR_TEXTURE(renderingTexture)); + // For some reason, deleting the outputTexture when cleaning the world causes crash on macos when closing the app. +#ifndef __APPLE__ + WrTextureRtt *outputTexture = wr_frame_buffer_get_output_texture(mResultFrameBuffer, 1); wr_texture_delete(WR_TEXTURE(outputTexture)); +#endif + wr_frame_buffer_delete(mResultFrameBuffer); wr_texture_delete(WR_TEXTURE(mNoiseMaskTexture)); mNoiseMaskTexture = NULL; diff --git a/tests/api/controllers/robot_urdf/robot_urdf.c b/tests/api/controllers/robot_urdf/robot_urdf.c index 40fd216d5b5..a63945ba425 100644 --- a/tests/api/controllers/robot_urdf/robot_urdf.c +++ b/tests/api/controllers/robot_urdf/robot_urdf.c @@ -60,7 +60,9 @@ int main(int argc, char **argv) { break; usleep(100000); // sleep for 100 ms }; - ts_assert_int_not_equal(file_size, 0, "check_urdf command is missing"); + ts_assert_int_not_equal(file_size, 0, + "check_urdf command is missing. " + "Install with 'apt install liburdfdom-tools' on Ubuntu or 'brew install urdfdom' on MacOS."); if ((urdf_check_status >> 8) != 127 && file_size > 0) { // `check_urdf` command is available // Verify output from `check_urdf` diff --git a/tests/test_suite.py b/tests/test_suite.py index 582466ff666..2fff35b42e9 100755 --- a/tests/test_suite.py +++ b/tests/test_suite.py @@ -350,20 +350,20 @@ def runGroupTest(groupName, firstSimulation, worldsCount, failures): webotsEmptyWorldPath], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) atexit.register(subprocess.Popen.terminate, self=backgroundWebots) # Wait until we can actually connect to it, trying 10 times. -with contextlib.closing(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) as sock: - retries = 0 - error = None - while retries < 10: - try: +retries = 0 +error = None +while retries < 10: + try: + with contextlib.closing(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) as sock: sock.settimeout(1) sock.connect(("127.0.0.1", 1234)) break - except socket.error as e: - error = e - retries += 1 - time.sleep(1) - if retries == 10: - raise error + except socket.error as e: + error = e + retries += 1 + time.sleep(1) +if retries == 10: + raise error for groupName in testGroups: if groupName == 'cache': From 3ec656004f36c665bf6e369252d1a22a1d81b864 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Mon, 26 Aug 2024 06:23:05 -0700 Subject: [PATCH 08/19] Fix bugs (introduced in PR #6622) when grabbing/recording with devicePixelRatio != 1. (#6638) * Fix bug (introduced in PR #6622) when grabbing with devicePixelRatio != 1. * Fix video recording size bug introduced by PR #6622. --- src/webots/gui/WbVideoRecorder.cpp | 7 +++---- src/webots/gui/WbWrenWindow.cpp | 12 +++++------- 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/src/webots/gui/WbVideoRecorder.cpp b/src/webots/gui/WbVideoRecorder.cpp index 5d10e581af6..194d831d88e 100644 --- a/src/webots/gui/WbVideoRecorder.cpp +++ b/src/webots/gui/WbVideoRecorder.cpp @@ -345,8 +345,7 @@ void WbVideoRecorder::stopRecording(bool canceled) { void WbVideoRecorder::writeSnapshot(unsigned char *frame) { QString fileName = nextFileName(); - FrameWriterThread *thread = - new FrameWriterThread(frame, fileName, mVideoResolution * mScreenPixelRatio, mScreenPixelRatio, mVideoQuality); + FrameWriterThread *thread = new FrameWriterThread(frame, fileName, mVideoResolution, 1, mVideoQuality); connect(thread, &QThread::finished, this, &WbVideoRecorder::terminateSnapshotWrite); thread->start(); } @@ -480,8 +479,8 @@ void WbVideoRecorder::createMpeg() { if (ffmpegScript.open(QIODevice::WriteOnly)) { // bitrate range between 4 and 24000000 // cast into 'long long int' is mandatory on 32-bit machine - long long int bitrate = (long long int)mVideoQuality * mMovieFPS * mVideoResolution.width() * mVideoResolution.height() / - 256 / (mScreenPixelRatio * mScreenPixelRatio); + long long int bitrate = + (long long int)mVideoQuality * mMovieFPS * mVideoResolution.width() * mVideoResolution.height() / 256; QTextStream stream(&ffmpegScript); #ifndef _WIN32 diff --git a/src/webots/gui/WbWrenWindow.cpp b/src/webots/gui/WbWrenWindow.cpp index dba767bb7e8..18239f36ef8 100644 --- a/src/webots/gui/WbWrenWindow.cpp +++ b/src/webots/gui/WbWrenWindow.cpp @@ -313,12 +313,11 @@ QImage WbWrenWindow::grabWindowBufferNow() { mSnapshotBufferHeight = destinationHeight; mSnapshotBuffer = new unsigned char[4 * destinationWidth * destinationHeight]; } - const qreal ratio = devicePixelRatio(); - const int sourceWidth = destinationWidth * ratio; - const int sourceHeight = destinationHeight * ratio; + const int sourceWidth = destinationWidth; + const int sourceHeight = destinationHeight; unsigned char *temp = new unsigned char[4 * sourceWidth * sourceHeight]; readPixels(sourceWidth, sourceHeight, GL_BGRA, temp); - flipAndScaleDownImageBuffer(temp, mSnapshotBuffer, sourceWidth, sourceHeight, ratio); + flipAndScaleDownImageBuffer(temp, mSnapshotBuffer, sourceWidth, sourceHeight, 1.0); delete[] temp; WbWrenOpenGlContext::doneWren(); @@ -328,9 +327,8 @@ QImage WbWrenWindow::grabWindowBufferNow() { void WbWrenWindow::initVideoPBO() { WbWrenOpenGlContext::makeWrenCurrent(); - const int ratio = (int)devicePixelRatio(); - mVideoWidth = width() * ratio; - mVideoHeight = height() * ratio; + mVideoWidth = width(); + mVideoHeight = height(); const int size = 4 * mVideoWidth * mVideoHeight; wr_scene_init_frame_capture(wr_scene_get_instance(), PBO_COUNT, mVideoPBOIds, size); mVideoPBOIndex = -1; From d9f6f7b3350f9960b5362bb0b5e1bca81746b153 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Mon, 26 Aug 2024 17:38:01 -0700 Subject: [PATCH 09/19] Fix-to-include-system-openal (#6640) * Make openal-dev a compilation dependency. * Include system openal lib in webots instead of downloading from cyberbotics. * Remove OPENAL_INCLUDE from Makefile because it should be in the system include path. --- dependencies/Makefile.linux | 18 ++++-------------- .../install/linux_compilation_dependencies.sh | 2 +- src/webots/Makefile | 5 ++--- 3 files changed, 7 insertions(+), 18 deletions(-) diff --git a/dependencies/Makefile.linux b/dependencies/Makefile.linux index 27c0f66dad0..d13c0529787 100644 --- a/dependencies/Makefile.linux +++ b/dependencies/Makefile.linux @@ -8,7 +8,6 @@ DEPENDENCIES_URL = https://cyberbotics.com/files/repository/dependencies/linux64 QT_VERSION = 6.5.3 QT_PACKAGE = webots-qt-$(QT_VERSION)-linux64-release.tar.bz2 -OPENAL_PACKAGE = openal-linux64-1.16.0.tar.bz2 OIS_PACKAGE = libOIS.1.4.tar.bz2 ASSIMP_PACKAGE = libassimp-5.2.3.tar.bz2 PICO_PACKAGE = libpico.tar.bz2 @@ -58,22 +57,13 @@ $(WEBOTS_DEPENDENCY_PATH)/$(QT_PACKAGE): open-al-clean: - @rm -rf $(WEBOTS_DEPENDENCY_PATH)/openal $(WEBOTS_DEPENDENCY_PATH)/$(OPENAL_PACKAGE)* $(WEBOTS_HOME_LIB)/libopenal.so* + @rm -rf $(WEBOTS_HOME_LIB)/libopenal.so* open-al: $(WEBOTS_HOME_LIB)/libopenal.so -$(WEBOTS_HOME_LIB)/libopenal.so: $(WEBOTS_DEPENDENCY_PATH)/openal - @cp -a $(WEBOTS_DEPENDENCY_PATH)/openal/build/libopenal.so* $(WEBOTS_HOME_LIB)/ - -$(WEBOTS_DEPENDENCY_PATH)/openal: - @echo "# downloading $(OPENAL_PACKAGE)" - @rm -f $(WEBOTS_DEPENDENCY_PATH)/$(OPENAL_PACKAGE) - @wget -qq $(DEPENDENCIES_URL)/$(OPENAL_PACKAGE) -P $(WEBOTS_DEPENDENCY_PATH) - @if [ "$$(md5sum $(OPENAL_PACKAGE) | awk '{print $$1;}')" != "6c7c9a77dec67f42c51d0f035a94a090" ]; then echo "MD5 checksum failed for $(OPENAL_PACKAGE)"; exit 1; fi - @echo "# uncompressing $(OPENAL_PACKAGE)" - @tar xfm $(WEBOTS_DEPENDENCY_PATH)/$(OPENAL_PACKAGE) -C $(WEBOTS_DEPENDENCY_PATH) - @rm -f $(WEBOTS_DEPENDENCY_PATH)/$(OPENAL_PACKAGE) - +.LIBPATTERNS = lib%.so lib%.so.1 lib%.a +$(WEBOTS_HOME_LIB)/libopenal.so: -lopenal + @cp -a $<* $(WEBOTS_HOME_LIB)/ ois-clean: @rm -rf $(WEBOTS_DEPENDENCY_PATH)/$(OIS_PACKAGE) $(WEBOTS_HOME_LIB)/libOIS* $(WEBOTS_HOME)/include/libOIS diff --git a/scripts/install/linux_compilation_dependencies.sh b/scripts/install/linux_compilation_dependencies.sh index acb2fe6bece..90c17a42256 100755 --- a/scripts/install/linux_compilation_dependencies.sh +++ b/scripts/install/linux_compilation_dependencies.sh @@ -12,7 +12,7 @@ fi alias apt='apt --option="APT::Acquire::Retries=3"' apt update -apt install --yes git lsb-release cmake swig libglu1-mesa-dev libglib2.0-dev libfreeimage3 libfreetype6-dev libxml2-dev libboost-dev libssh-gcrypt-dev libzip-dev libreadline-dev pbzip2 wget zip unzip python3 python3-pip +apt install --yes git lsb-release cmake swig libglu1-mesa-dev libglib2.0-dev libfreeimage3 libfreetype6-dev libxml2-dev libboost-dev libssh-gcrypt-dev libzip-dev libreadline-dev pbzip2 wget zip unzip python3 python3-pip libopenal-dev UBUNTU_VERSION=$(lsb_release -rs) if [[ $UBUNTU_VERSION == "20.04" ]]; then diff --git a/src/webots/Makefile b/src/webots/Makefile index f0b2a0987c2..33a983976a3 100644 --- a/src/webots/Makefile +++ b/src/webots/Makefile @@ -44,7 +44,7 @@ WB_CONTROL_INCLUDE = $(QT_CORE_INCLUDE) $(QT_NETWORK_INCLUDE) -Icore -Ieng WB_NODES_INCLUDE = $(QT_CORE_INCLUDE) $(QT_NETWORK_INCLUDE) $(QT_GUI_INCLUDE) $(ODE_INCLUDE) $(CONTROLLER_INCLUDE) $(OIS_INCLUDE) $(FREETYPE_INCLUDE) $(WREN_INCLUDE) $(STB_INCLUDE) $(ASSIMP_INCLUDE) -Iapp -Icore -isystem external/siphash -Imaths -Inodes -Inodes/utils -Iode -Iplugins -Isound -Iutil -Ivrml WB_APP_INCLUDE = $(QT_CORE_INCLUDE) $(QT_NETWORK_INCLUDE) $(ODE_INCLUDE) $(WREN_INCLUDE) -Icontrol -Icore -Ieditor -Iengine -Imaths -Inodes -Inodes/utils -Iplugins -Iscene_tree -Ivrml WB_SCENE_TREE_INCLUDE = $(QT_CORE_INCLUDE) $(QT_GUI_INCLUDE) $(QT_WIDGETS_INCLUDE) $(ASSIMP_INCLUDE) -isystem external/siphash -Iapp -Icore -Ieditor -Imaths -Inodes -Inodes/utils -Iode -Iuser_commands -Ivrml -Iwidgets -WB_SOUND_INCLUDE = $(QT_CORE_INCLUDE) $(QT_XML_INCLUDE) $(ODE_INCLUDE) $(OPENAL_INCLUDE) $(PICO_INCLUDE) -Icore -Imaths -Inodes -Inodes/utils -Isound -Ivrml +WB_SOUND_INCLUDE = $(QT_CORE_INCLUDE) $(QT_XML_INCLUDE) $(ODE_INCLUDE) $(PICO_INCLUDE) -Icore -Imaths -Inodes -Inodes/utils -Isound -Ivrml WB_GUI_INCLUDE = $(QT_CORE_INCLUDE) $(QT_NETWORK_INCLUDE) $(QT_WEBSOCKETS_INCLUDE) $(QT_GUI_INCLUDE) $(QT_WIDGETS_INCLUDE) $(QT_OPENGL_INCLUDE) $(QT_OPENGL_WIDGETS_INCLUDE) $(ODE_INCLUDE) $(WREN_INCLUDE) -Iapp -Icontrol -Icore -Iwidgets -Ieditor -Iengine -isystem external/siphash -Imaths -Inodes -Inodes/utils -Iplugins -Iscene_tree -Isound -Iuser_commands -Iutil -Ivrml WB_RENDER_INCLUDE = $(QT_CORE_INCLUDE) $(QT_GUI_INCLUDE) $(QT_OPENGL_INCLUDE) $(QT_WIDGETS_INCLUDE) $(WREN_INCLUDE) -isystem external/siphash @@ -104,7 +104,6 @@ CFLAGS += -Wno-deprecated-register else ifeq ($(OSTYPE),linux) LIB_WREN = ../wren/libwren.a ../glad/libglad.a -OPENAL_INCLUDE = -I$(WEBOTS_DEPENDENCY_PATH)/openal/include ASSIMP_INCLUDE = -I$(WEBOTS_PATH)/include/libassimp/include QT_CORE_INCLUDE = -isystem $(WEBOTS_PATH)/include/qt/QtCore QT_GUI_INCLUDE = -isystem $(WEBOTS_PATH)/include/qt/QtGui @@ -182,7 +181,7 @@ CONTROLLER_INCLUDE = -I$(WEBOTS_PATH)/include/controller/c ODE_INCLUDE = -Iode -isystem $(WEBOTS_PATH)/include/ode QT_CORE_INCLUDE += -I$(OBJDIR) WREN_INCLUDE += -Iwren -INCLUDE = $(ALL_INCLUDE) $(CONTROLLER_INCLUDE) $(ODE_INCLUDE) $(QT_CORE_INCLUDE) $(QT_GUI_INCLUDE) $(QT_WIDGETS_INCLUDE) $(QT_PRINT_SUPPORT_INCLUDE) $(QT_OPENGL_INCLUDE) $(QT_NETWORK_INCLUDE) $(LUA_INCLUDE) $(OIS_INCLUDE) $(OPENAL_INCLUDE) $(PICO_INCLUDE) $(WREN_INCLUDE) $(FREETYPE_INCLUDE) $(STB_INCLUDE) $(ASSIMP_INCLUDE) +INCLUDE = $(ALL_INCLUDE) $(CONTROLLER_INCLUDE) $(ODE_INCLUDE) $(QT_CORE_INCLUDE) $(QT_GUI_INCLUDE) $(QT_WIDGETS_INCLUDE) $(QT_PRINT_SUPPORT_INCLUDE) $(QT_OPENGL_INCLUDE) $(QT_NETWORK_INCLUDE) $(LUA_INCLUDE) $(OIS_INCLUDE) $(PICO_INCLUDE) $(WREN_INCLUDE) $(FREETYPE_INCLUDE) $(STB_INCLUDE) $(ASSIMP_INCLUDE) ifeq ($(MAKECMDGOALS),debug) CFLAGS += -ggdb From ea411966eff856c441d1610cbfd76632dc8b7f32 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Tue, 27 Aug 2024 13:52:29 -0700 Subject: [PATCH 10/19] Fix failure of supervisor_start_stop_movie introduced by PR #6623. (#6645) * Fix failure of supervisor_start_stop_movie introduced by PR #6623. If the system ffmpeg links (possibly indirectly) to a different openssl than what we bundle, ffmpeg will fail to start. This fixes that by restoring the original LD_LIBRARY_PATH while running ffmpeg. --- src/webots/gui/WbVideoRecorder.cpp | 2 +- src/webots/launcher/webots-linux.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/src/webots/gui/WbVideoRecorder.cpp b/src/webots/gui/WbVideoRecorder.cpp index 194d831d88e..a98b7513711 100644 --- a/src/webots/gui/WbVideoRecorder.cpp +++ b/src/webots/gui/WbVideoRecorder.cpp @@ -459,7 +459,7 @@ QString WbVideoRecorder::nextFileName() { void WbVideoRecorder::createMpeg() { #ifdef __linux__ - static const QString ffmpeg("ffmpeg"); + static const QString ffmpeg("LD_LIBRARY_PATH=\"$WEBOTS_ORIGINAL_LD_LIBRARY_PATH\" ffmpeg"); static const QString percentageChar = "%"; mScriptPath = "ffmpeg_script.sh"; #elif defined(__APPLE__) diff --git a/src/webots/launcher/webots-linux.sh b/src/webots/launcher/webots-linux.sh index f489716ad3d..40b9159f077 100644 --- a/src/webots/launcher/webots-linux.sh +++ b/src/webots/launcher/webots-linux.sh @@ -69,6 +69,7 @@ export TMPDIR=$WEBOTS_TMPDIR export WEBOTS_TMPDIR=$WEBOTS_TMPDIR # add the "lib" directory into LD_LIBRARY_PATH as the first entry +export WEBOTS_ORIGINAL_LD_LIBRARY_PATH="$LD_LIBRARY_PATH" export LD_LIBRARY_PATH="$webots_home/lib/webots":$LD_LIBRARY_PATH # Fix for i3 window manager not working with Qt6 From 8d0ecd797700d307de137e99b0d0575f400a5a8a Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Tue, 27 Aug 2024 15:44:07 -0700 Subject: [PATCH 11/19] Fix-mirror-crash-under-wayland (#6644) * Use format() instead of cMainOpenGLContext->format() or requestedFormat() when creating context that will be shared. * Create rendering device window with the same format as used by the main OpenGLContext so that they are compatible. --- src/webots/gui/WbRenderingDeviceWindow.cpp | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/webots/gui/WbRenderingDeviceWindow.cpp b/src/webots/gui/WbRenderingDeviceWindow.cpp index 1efe540efc1..8d954ea5a16 100644 --- a/src/webots/gui/WbRenderingDeviceWindow.cpp +++ b/src/webots/gui/WbRenderingDeviceWindow.cpp @@ -112,9 +112,8 @@ WbRenderingDeviceWindow::WbRenderingDeviceWindow(WbRenderingDevice *device) : mUpdateRequested(true), mShowOverlayOnClose(true) { setSurfaceType(QWindow::OpenGLSurface); - QSurfaceFormat surfaceFormat = format(); - surfaceFormat.setMajorVersion(3); - surfaceFormat.setMinorVersion(3); + assert(cMainOpenGLContext); + QSurfaceFormat surfaceFormat = cMainOpenGLContext->format(); setFormat(surfaceFormat); mAbstractCamera = dynamic_cast(mDevice); From 2a0c4ead4f46d5d4fbc5ac3c9f849c862bb87b46 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Sat, 31 Aug 2024 06:41:08 -0500 Subject: [PATCH 12/19] Fix-test-suite-not-run-on-macos (#6631) Fixes to run test_suite.yml during macos CI (if "test suite mac" label is present). * Make sure the Contents/Frameworks folder is created. * Set WEBOTS_HOME_PATH such that tests can find Makefile.include during CI. * Change WEBOTS_HOME to Webots.app and add logic to test_suite.py to find webots executable. * Get qt_warning_filters.conf from correct location on macos. * Ignore (but warn about) error trying to connect to background Webots on MacOS. * On Mac, timeout test after 60 minutes instead of 10. * Use macos-13 runner instead of macos-14. The former has twice as much RAM as the latter. * Install urdfdom during macos CI because it is needed by tests suite. * Assert that move succeeds. * Print age of ode_tmp.dif so we can see how long we should wait on macos during CI. * Allow up to 20 seconds for world reload. * Skip billboard.wbt test during macos CI. The test passes locally, so the failure is presumably due to the CI server being headless. * Only increase timeout on macos. --------- Co-authored-by: CoolSpy3 <55305038+CoolSpy3@users.noreply.github.com> Co-authored-by: Olivier Michel --- .github/workflows/test_suite_mac.yml | 20 +++++++++++-------- src/webots/gui/main.cpp | 6 +++++- .../test_suite_supervisor.py | 3 +++ .../controllers/determinism/determinism.c | 6 +++--- tests/test_suite.py | 12 +++++++++-- 5 files changed, 33 insertions(+), 14 deletions(-) diff --git a/.github/workflows/test_suite_mac.yml b/.github/workflows/test_suite_mac.yml index aa526a8d088..90206863f68 100644 --- a/.github/workflows/test_suite_mac.yml +++ b/.github/workflows/test_suite_mac.yml @@ -19,7 +19,7 @@ jobs: if: ${{ contains(github.event.pull_request.labels.*.name, 'test webots build') }} strategy: matrix: - os: [macos-14] + os: [macos-13] runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v3 @@ -33,7 +33,7 @@ jobs: if: ${{ github.event_name == 'push' || github.event_name == 'schedule' || contains(github.event.pull_request.labels.*.name, 'test distribution') || contains(github.event.pull_request.labels.*.name, 'test suite') }} strategy: matrix: - os: [macos-14] + os: [macos-13] runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v3 @@ -67,10 +67,10 @@ jobs: distribution/*.dmg test-suite: needs: build - if: ${{ contains(github.event.pull_request.labels.*.name, 'test suite') }} + if: ${{ contains(github.event.pull_request.labels.*.name, 'test suite mac') }} strategy: matrix: - os: [macos-14] + os: [macos-13] runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v3 @@ -89,17 +89,21 @@ jobs: with: python-version: 3.11 - name: Test - run: + run: | + brew install urdfdom export WEBOTS_DISABLE_SAVE_SCREEN_PERSPECTIVE_ON_CLOSE=true + python scripts/packaging/update_urls.py $(git log -1 --format='%H') export WEBOTS_HOME=/Applications/Webots.app - # we currently can't run the test suite because of https://github.com/cyberbotics/webots/issues/2110 - # python tests/test_suite.py + export TESTS_HOME=$PWD # required by cache group in the test suite + export WEBOTS_HOME_PATH=$PWD # required for tests to find Makefile.include + export BRANCH_HASH=$(git log -1 --format='%H') + python tests/test_suite.py delete-artifacts: needs: [build, test-suite] if: ${{ always() && !contains(github.event.pull_request.labels.*.name, 'test distribution') && !contains(github.event.pull_request.labels.*.name, 'test webots build') }} strategy: matrix: - os: [macos-14] + os: [macos-13] runs-on: ubuntu-latest steps: - name: Delete artifacts diff --git a/src/webots/gui/main.cpp b/src/webots/gui/main.cpp index 84a8a6d736c..2b49b9189b8 100644 --- a/src/webots/gui/main.cpp +++ b/src/webots/gui/main.cpp @@ -147,8 +147,12 @@ int main(int argc, char *argv[]) { #endif ); - // load qt warning filters from file +#ifdef __APPLE__ + QString qtFiltersFilePath = QDir::fromNativeSeparators(webotsDirPath + "/Contents/Resources/qt_warning_filters.conf"); +#else QString qtFiltersFilePath = QDir::fromNativeSeparators(webotsDirPath + "/resources/qt_warning_filters.conf"); +#endif + // load qt warning filters from file QFile qtFiltersFile(qtFiltersFilePath); if (qtFiltersFile.open(QIODevice::ReadOnly)) { QTextStream in(&qtFiltersFile); diff --git a/tests/default/controllers/test_suite_supervisor/test_suite_supervisor.py b/tests/default/controllers/test_suite_supervisor/test_suite_supervisor.py index fafbd52126f..59889494f0f 100644 --- a/tests/default/controllers/test_suite_supervisor/test_suite_supervisor.py +++ b/tests/default/controllers/test_suite_supervisor/test_suite_supervisor.py @@ -240,6 +240,9 @@ def run(self): # 30 seconds before executing the next world, 60 seconds for the robot_window_html test delay = 60 if self.currentSimulationFilename.endswith('/robot_window_html.wbt') else 30 + if sys.platform == "darwin": + # Longer timeout on MacOS because Webots takes longer to start there during CI. + delay *= 10 timeout = time.time() + delay running_controllers_pid = [] diff --git a/tests/physics/controllers/determinism/determinism.c b/tests/physics/controllers/determinism/determinism.c index 764953a2116..e91ac27cb1e 100644 --- a/tests/physics/controllers/determinism/determinism.c +++ b/tests/physics/controllers/determinism/determinism.c @@ -28,8 +28,8 @@ int main(int argc, char **argv) { if (file_exists(ode_tmp_dif)) { double delta = difftime(now, file_get_creation_time(ode_tmp_dif)); - // printf("delta time = %f\n", delta); - if (delta > 2.0) { + // printf("%s is %f seconds old.\n", ode_tmp_dif, delta); + if (delta > 20.0) { ts_assert_boolean_equal(remove_file(ode_tmp_dif), "Cannot remove ODE tmp dif file"); ts_assert_boolean_not_equal(file_exists(ode_tmp_dif), "Cannot remove ODE tmp dif file"); } @@ -60,7 +60,7 @@ int main(int argc, char **argv) { exit(EXIT_FAILURE); } } else { - move_file(ode_dif, ode_tmp_dif); + ts_assert_boolean_equal(move_file(ode_dif, ode_tmp_dif), "Cannot move ODF dif file to ODE tmp dif file"); wb_supervisor_world_reload(); wb_robot_cleanup(); exit(EXIT_SUCCESS); diff --git a/tests/test_suite.py b/tests/test_suite.py index 2fff35b42e9..dac4d398c09 100755 --- a/tests/test_suite.py +++ b/tests/test_suite.py @@ -86,6 +86,8 @@ def setupWebots(): if sys.platform == 'win32': webotsFullPath = os.path.join(os.path.normpath(os.environ['WEBOTS_HOME']), 'msys64', 'mingw64', 'bin', 'webots.exe') + elif sys.platform == 'darwin': + webotsFullPath = os.path.join(os.path.normpath(os.environ['WEBOTS_HOME']), 'Contents', 'MacOS', 'webots') else: webotsBinary = 'webots' if 'WEBOTS_HOME' in os.environ: @@ -198,12 +200,14 @@ def generateWorldsList(groupName): for filename in filenames: # speaker test not working on github action because of missing sound drivers # robot window and movie recording test not working on BETA Ubuntu 22.04 GitHub Action environment + # billboard test not working in macos GitHub Action environment if (not filename.endswith('_temp.wbt') and not ('GITHUB_ACTIONS' in os.environ and ( filename.endswith('speaker.wbt') or filename.endswith('local_proto_with_texture.wbt') or (filename.endswith('robot_window_html.wbt') and is_ubuntu_22_04) or - (filename.endswith('supervisor_start_stop_movie.wbt') and is_ubuntu_22_04) + (filename.endswith('supervisor_start_stop_movie.wbt') and is_ubuntu_22_04) or + (filename.endswith('billboard.wbt') and sys.platform == 'darwin') ))): worldsList.append(filename) @@ -244,7 +248,11 @@ def runGroupTest(groupName, firstSimulation, worldsCount, failures): command = Command(webotsArguments) # redirect stdout and stderr to files - command.runTest(timeout=10 * 60) # 10 minutes + timeoutMinutes = 10 + if sys.platform == "darwin": + # Longer timeout on MacOS because Webots takes longer to start there during CI. + timeoutMinutes = 60 + command.runTest(timeout=timeoutMinutes * 60) if command.isTimeout or command.returncode != 0: if command.isTimeout: From eed811b91af3825ebfa3ad0ebcdaffb6e7f5dd38 Mon Sep 17 00:00:00 2001 From: CoolSpy3 <55305038+CoolSpy3@users.noreply.github.com> Date: Sun, 1 Sep 2024 10:59:46 -0700 Subject: [PATCH 13/19] Fix Crash When `wb_supervisor_field_get_name` is Called With NULL (#6647) * fix crash when wb_supervisor_field_get_name is called with NULL * update changelog * run clang-format * use past-tense in changelog * add period to changelog Co-authored-by: Olivier Michel --------- Co-authored-by: Olivier Michel --- docs/reference/changelog-r2024.md | 1 + src/controller/c/supervisor.c | 3 +++ tests/api/controllers/supervisor_field/supervisor_field.c | 7 +++++++ 3 files changed, 11 insertions(+) diff --git a/docs/reference/changelog-r2024.md b/docs/reference/changelog-r2024.md index e27c7cb6361..16c867eb9bc 100644 --- a/docs/reference/changelog-r2024.md +++ b/docs/reference/changelog-r2024.md @@ -28,3 +28,4 @@ Released on December **th, 2023. - Fixed connection errors when using long robot names in some environments ([#6635](https://github.com/cyberbotics/webots/pull/6635)). - Fixed crash on macos when closing Webots under some circumstances ([#6635](https://github.com/cyberbotics/webots/pull/6635)). - Fixed error on macos when when putting displays and cameras in separate windows ([#6635](https://github.com/cyberbotics/webots/pull/6635)). + - Fixed crash when `wb_supervisor_field_get_name` was called with NULL ([#6647](https://github.com/cyberbotics/webots/pull/6647)). diff --git a/src/controller/c/supervisor.c b/src/controller/c/supervisor.c index b20574914ce..e7a6dc47a19 100644 --- a/src/controller/c/supervisor.c +++ b/src/controller/c/supervisor.c @@ -2831,6 +2831,9 @@ const double *wb_supervisor_virtual_reality_headset_get_orientation() { } const char *wb_supervisor_field_get_name(WbFieldRef field) { + if (!check_field(field, __FUNCTION__, WB_NO_FIELD, false, NULL, false, false)) + return ""; + return field->name; } diff --git a/tests/api/controllers/supervisor_field/supervisor_field.c b/tests/api/controllers/supervisor_field/supervisor_field.c index 5df8e00cb41..3e5bbfe0265 100644 --- a/tests/api/controllers/supervisor_field/supervisor_field.c +++ b/tests/api/controllers/supervisor_field/supervisor_field.c @@ -25,6 +25,13 @@ int main(int argc, char **argv) { int i; double d; + ts_assert_string_equal(wb_supervisor_field_get_name(NULL), "", + "wb_supervisor_field_get_name(NULL) should return the empty string"); + ts_assert_int_equal(wb_supervisor_field_get_type(NULL), 0, "wb_supervisor_field_get_type(NULL) should return 0"); + ts_assert_string_equal(wb_supervisor_field_get_type_name(NULL), "", + "wb_supervisor_field_get_type_name(NULL) should return the empty string"); + ts_assert_int_equal(wb_supervisor_field_get_count(NULL), -1, "wb_supervisor_field_get_count(NULL) should return -1"); + root = wb_supervisor_node_get_root(); ts_assert_pointer_not_null(root, "Root node is not found"); From 4501943232712d2df5ebc9e2df2eef31f766673a Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 4 Sep 2024 07:45:58 +0200 Subject: [PATCH 14/19] Bump actions/download-artifact from 3 to 4.1.7 in /.github/workflows (#6652) Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4.1.7. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](https://github.com/actions/download-artifact/compare/v3...v4.1.7) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/test_suite_linux.yml | 6 +++--- .github/workflows/test_suite_linux_develop.yml | 6 +++--- .github/workflows/test_suite_mac.yml | 2 +- .github/workflows/test_suite_mac_develop.yml | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/.github/workflows/test_suite_linux.yml b/.github/workflows/test_suite_linux.yml index 0cbb501183e..8a957edc7cd 100644 --- a/.github/workflows/test_suite_linux.yml +++ b/.github/workflows/test_suite_linux.yml @@ -100,7 +100,7 @@ jobs: steps: - uses: actions/checkout@v3 - name: Download Artifacts - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4.1.7 with: name: build-ubuntu-20.04 path: artifact @@ -144,7 +144,7 @@ jobs: with: submodules: true - name: Download Artifacts - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4.1.7 with: name: build-${{ matrix.os }} path: artifact @@ -171,7 +171,7 @@ jobs: runs-on: ubuntu-20.04 steps: - name: Download Artifacts - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4.1.7 with: name: build-ubuntu-20.04 path: artifact diff --git a/.github/workflows/test_suite_linux_develop.yml b/.github/workflows/test_suite_linux_develop.yml index e205d96c4e3..e3629e8b868 100644 --- a/.github/workflows/test_suite_linux_develop.yml +++ b/.github/workflows/test_suite_linux_develop.yml @@ -98,7 +98,7 @@ jobs: with: ref: develop - name: Download Artifacts - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4.1.7 with: name: build-ubuntu-20.04 path: artifact @@ -137,7 +137,7 @@ jobs: submodules: true ref: develop - name: Download Artifacts - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4.1.7 with: name: build-${{ matrix.os }} path: artifact @@ -164,7 +164,7 @@ jobs: runs-on: ubuntu-20.04 steps: - name: Download Artifacts - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4.1.7 with: name: build-ubuntu-20.04 path: artifact diff --git a/.github/workflows/test_suite_mac.yml b/.github/workflows/test_suite_mac.yml index 90206863f68..010ea538393 100644 --- a/.github/workflows/test_suite_mac.yml +++ b/.github/workflows/test_suite_mac.yml @@ -75,7 +75,7 @@ jobs: steps: - uses: actions/checkout@v3 - name: Download Artifacts - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4.1.7 with: name: build-${{ matrix.os }} path: artifact diff --git a/.github/workflows/test_suite_mac_develop.yml b/.github/workflows/test_suite_mac_develop.yml index 07eca215b29..a619b9fa0be 100644 --- a/.github/workflows/test_suite_mac_develop.yml +++ b/.github/workflows/test_suite_mac_develop.yml @@ -73,7 +73,7 @@ jobs: with: ref: develop - name: Download Artifacts - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4.1.7 with: name: build-${{ matrix.os }} path: artifact From ebf8625814778672c2806346198e3532c887610f Mon Sep 17 00:00:00 2001 From: CoolSpy3 <55305038+CoolSpy3@users.noreply.github.com> Date: Thu, 5 Sep 2024 21:54:21 -0700 Subject: [PATCH 15/19] Bump actions/upload-artifact to v4 (#6654) --- .github/workflows/test_suite_linux.yml | 6 +++--- .github/workflows/test_suite_linux_develop.yml | 4 ++-- .github/workflows/test_suite_mac.yml | 2 +- .github/workflows/test_suite_mac_develop.yml | 2 +- .github/workflows/test_suite_windows.yml | 2 +- .github/workflows/test_suite_windows_develop.yml | 2 +- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/.github/workflows/test_suite_linux.yml b/.github/workflows/test_suite_linux.yml index 8a957edc7cd..130d5cb91b4 100644 --- a/.github/workflows/test_suite_linux.yml +++ b/.github/workflows/test_suite_linux.yml @@ -74,14 +74,14 @@ jobs: sudo python -m pip install requests PyGithub pip install pyopenssl --upgrade scripts/packaging/publish_release.py --key=${{ secrets.GITHUB_TOKEN }} --repo=${{ github.repository }} --branch=${{ github.ref }} --commit=$(git log -1 --format='%H') --tag=${{ github.ref }} - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: ${{ contains(github.event.pull_request.labels.*.name, 'test suite') || contains(github.event.pull_request.labels.*.name, 'test ros') || contains(github.event.pull_request.labels.*.name, 'test worlds') }} with: name: build-${{ matrix.os }} path: | distribution/*.tar.bz2 distribution/*.zip - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: ${{ !contains(github.event.pull_request.labels.*.name, 'test suite') && !contains(github.event.pull_request.labels.*.name, 'test ros') && !contains(github.event.pull_request.labels.*.name, 'test worlds') }} with: name: build-${{ matrix.os }} @@ -121,7 +121,7 @@ jobs: export TESTS_HOME=$PWD # required by cache group in the test suite export BRANCH_HASH=$(git log -1 --format='%H') xvfb-run --auto-servernum python tests/test_suite.py - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: failure() with: name: test-results-${{ matrix.os }} diff --git a/.github/workflows/test_suite_linux_develop.yml b/.github/workflows/test_suite_linux_develop.yml index e3629e8b868..d365ab95b66 100644 --- a/.github/workflows/test_suite_linux_develop.yml +++ b/.github/workflows/test_suite_linux_develop.yml @@ -70,14 +70,14 @@ jobs: sudo python -m pip install requests PyGithub pip install pyopenssl --upgrade scripts/packaging/publish_release.py --key=${{ secrets.GITHUB_TOKEN }} --repo=${{ github.repository }} --branch=${{ github.ref }} --commit=$(git log -1 --format='%H') --tag=${{ github.ref }} - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: ${{ contains(github.event.pull_request.labels.*.name, 'test suite') || contains(github.event.pull_request.labels.*.name, 'test ros') || contains(github.event.pull_request.labels.*.name, 'test worlds') }} with: name: build-${{ matrix.os }} path: | distribution/*.tar.bz2 distribution/*.zip - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: ${{ !contains(github.event.pull_request.labels.*.name, 'test suite') && !contains(github.event.pull_request.labels.*.name, 'test ros') && !contains(github.event.pull_request.labels.*.name, 'test worlds') }} with: name: build-${{ matrix.os }} diff --git a/.github/workflows/test_suite_mac.yml b/.github/workflows/test_suite_mac.yml index 010ea538393..175ee3177ac 100644 --- a/.github/workflows/test_suite_mac.yml +++ b/.github/workflows/test_suite_mac.yml @@ -59,7 +59,7 @@ jobs: run: | sudo python -m pip install requests PyGithub scripts/packaging/publish_release.py --key=${{ secrets.GITHUB_TOKEN }} --repo=${{ github.repository }} --branch=${{ github.ref }} --commit=$(git log -1 --format='%H') --tag=${{ github.ref }} - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: ${{ contains(github.event.pull_request.labels.*.name, 'test suite') || contains(github.event.pull_request.labels.*.name, 'test distribution') }} with: name: build-${{ matrix.os }} diff --git a/.github/workflows/test_suite_mac_develop.yml b/.github/workflows/test_suite_mac_develop.yml index a619b9fa0be..bfe913eacc0 100644 --- a/.github/workflows/test_suite_mac_develop.yml +++ b/.github/workflows/test_suite_mac_develop.yml @@ -55,7 +55,7 @@ jobs: run: | sudo python -m pip install requests PyGithub scripts/packaging/publish_release.py --key=${{ secrets.GITHUB_TOKEN }} --repo=${{ github.repository }} --branch=${{ github.ref }} --commit=$(git log -1 --format='%H') --tag=${{ github.ref }} - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: ${{ contains(github.event.pull_request.labels.*.name, 'test suite') || contains(github.event.pull_request.labels.*.name, 'test distribution') }} with: name: build-${{ matrix.os }} diff --git a/.github/workflows/test_suite_windows.yml b/.github/workflows/test_suite_windows.yml index ad816213ddc..1127e4faeac 100644 --- a/.github/workflows/test_suite_windows.yml +++ b/.github/workflows/test_suite_windows.yml @@ -87,7 +87,7 @@ jobs: run: | python -m pip install requests PyGithub scripts/packaging/publish_release.py --key=${{ secrets.GITHUB_TOKEN }} --repo=${{ github.repository }} --branch=${{ github.ref }} --commit=$(git log -1 --format='%H') --tag=${{ github.ref }} - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: ${{ contains(github.event.pull_request.labels.*.name, 'test distribution') }} with: name: build-${{ matrix.os }} diff --git a/.github/workflows/test_suite_windows_develop.yml b/.github/workflows/test_suite_windows_develop.yml index 4e57d062c54..7327938545c 100644 --- a/.github/workflows/test_suite_windows_develop.yml +++ b/.github/workflows/test_suite_windows_develop.yml @@ -83,7 +83,7 @@ jobs: run: | python -m pip install requests PyGithub scripts/packaging/publish_release.py --key=${{ secrets.GITHUB_TOKEN }} --repo=${{ github.repository }} --branch=${{ github.ref }} --commit=$(git log -1 --format='%H') --tag=${{ github.ref }} - - uses: actions/upload-artifact@v3 + - uses: actions/upload-artifact@v4 if: ${{ contains(github.event.pull_request.labels.*.name, 'test distribution') }} with: name: build-${{ matrix.os }} From 34dfe53ffc29922e109d15313846d4ce00dfbe03 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Thu, 12 Sep 2024 05:50:53 -0700 Subject: [PATCH 16/19] Run test suite on windows during CI (#6648) * Add a test-suite job. * Upload artifact after build if test suite requested. * Setup msys2. * Update test_suite_windows.yml * Install deps. * Install VS10 and OpenJdk so test-suite can run. * Set WEBOTS_HOME. * Add WEBOTS_HOME to PATH before running tests. * Fix PATH. * Print sysinfo output when error occurs. * Print debug info when getting sysinfo. * Install software renderer for headless CI. * Launch tmate session for debugging. * Setup tmate session if the test suite fails. * Read MyBot.txt in text instead of binary mode to compensate for line ending differences. * Treat hdr files as binary so they aren't corrupted by git when checking out on Windows. * Use download artifact v4. * Override GL version to be 3.3 and force software rendering. * Skip a couple tests that don't work on windows CI. * Make test_suite.py accept world files under windows. --------- Co-authored-by: CoolSpy3 <55305038+CoolSpy3@users.noreply.github.com> Co-authored-by: Olivier Michel --- .github/workflows/test_suite_windows.yml | 66 ++++++++++++++++++- projects/.gitattributes | 1 + src/webots/gui/WbSingleTaskApplication.cpp | 5 +- .../supervisor_import_robot_node.c | 2 +- tests/test_suite.py | 9 ++- 5 files changed, 75 insertions(+), 8 deletions(-) create mode 100644 projects/.gitattributes diff --git a/.github/workflows/test_suite_windows.yml b/.github/workflows/test_suite_windows.yml index 1127e4faeac..96aae79b3f0 100644 --- a/.github/workflows/test_suite_windows.yml +++ b/.github/workflows/test_suite_windows.yml @@ -88,13 +88,75 @@ jobs: python -m pip install requests PyGithub scripts/packaging/publish_release.py --key=${{ secrets.GITHUB_TOKEN }} --repo=${{ github.repository }} --branch=${{ github.ref }} --commit=$(git log -1 --format='%H') --tag=${{ github.ref }} - uses: actions/upload-artifact@v4 - if: ${{ contains(github.event.pull_request.labels.*.name, 'test distribution') }} + if: ${{ contains(github.event.pull_request.labels.*.name, 'test suite') || contains(github.event.pull_request.labels.*.name, 'test distribution') }} with: name: build-${{ matrix.os }} path: | distribution/*.exe + test-suite: + needs: build + if: ${{ contains(github.event.pull_request.labels.*.name, 'test suite') }} + strategy: + matrix: + os: [windows-2019] + runs-on: ${{ matrix.os }} + steps: + - uses: actions/checkout@v3 + - name: Download Artifacts + uses: actions/download-artifact@v4 + with: + name: build-${{ matrix.os }} + path: artifact + - uses: msys2/setup-msys2@v2 + with: + update: false + - name: Install Visual Studio 10 and OpenJDK 18 + shell: powershell + run: | + choco install -y openjdk --version=18.0.2 + choco install -y visualcpp-build-tools + - name: Install Webots Compilation Dependencies + run: | + export PYTHON_INSTALLATION_FOLDER=/C/hostedtoolcache/windows/Python + export PYTHON_HOME=$PYTHON_INSTALLATION_FOLDER/3.11.`ls $PYTHON_INSTALLATION_FOLDER | grep '^3\.11\.[0-9]\+$' | cut -c6- | sort -n | tail -n1`/x64 + echo 'export JAVA_HOME=/C/Program\ Files/OpenJDK/`ls /C/Program\ Files/OpenJDK`' >> ~/.bash_profile + echo 'export PYTHON_HOME='$PYTHON_HOME >> ~/.bash_profile + echo 'export VISUAL_STUDIO_PATH="/C/Program Files (x86)/Microsoft Visual Studio/2017"' >> ~/.bash_profile + echo 'export INNO_SETUP_HOME="/C/Program Files (x86)/Inno Setup 6"' >> ~/.bash_profile + echo 'export PATH=$PYTHON_HOME:$PYTHON_HOME/Scripts:$GITHUB_WORKSPACE/msys64/mingw64/bin:$GITHUB_WORKSPACE/bin/node:/mingw64/bin:/usr/bin:$JAVA_HOME/bin:$PATH' >> ~/.bash_profile + export WEBOTS_HOME=$GITHUB_WORKSPACE + ./scripts/install/msys64_installer.sh --all + - name: Extract Webots + run: | + ./artifact/webots-*.exe //SUPPRESSMSGBOXES //VERYSILENT //NOCANCEL //NORESTART //ALLUSERS + - name: Set up Python 3.11 + uses: actions/setup-python@v4 + with: + python-version: 3.11 + - name: Install Software Renderer + uses: ssciwr/setup-mesa-dist-win@v2 + with: + version: '24.2.1' + build-type: 'release-mingw' + - name: Test + run: | + export WEBOTS_DISABLE_SAVE_SCREEN_PERSPECTIVE_ON_CLOSE=true + python scripts/packaging/update_urls.py $(git log -1 --format='%H') + export TESTS_HOME=$PWD # required by cache group in the test suite + export WEBOTS_HOME="/c/Program Files/Webots" + export PATH="${WEBOTS_HOME}/msys64/mingw64/bin:$PATH" + export WEBOTS_HOME_PATH=$PWD # required for tests to find Makefile.include + export BRANCH_HASH=$(git log -1 --format='%H') + export MESA_GL_VERSION_OVERRIDE=3.3 + export LIBGL_ALWAYS_SOFTWARE=true + python tests/test_suite.py + - name: Setup tmate session + if: ${{ failure() }} + uses: mxschmitt/action-tmate@v3 + with: + detached: true delete-artifacts: - needs: [build] + needs: [build, test-suite] if: ${{ always() && !contains(github.event.pull_request.labels.*.name, 'test distribution') && !contains(github.event.pull_request.labels.*.name, 'test webots build') }} strategy: matrix: diff --git a/projects/.gitattributes b/projects/.gitattributes new file mode 100644 index 00000000000..7cf76504658 --- /dev/null +++ b/projects/.gitattributes @@ -0,0 +1 @@ +*.hdr binary diff --git a/src/webots/gui/WbSingleTaskApplication.cpp b/src/webots/gui/WbSingleTaskApplication.cpp index 059d9dbdee6..067e05b45b6 100644 --- a/src/webots/gui/WbSingleTaskApplication.cpp +++ b/src/webots/gui/WbSingleTaskApplication.cpp @@ -231,10 +231,11 @@ void WbSingleTaskApplication::showSysInfo() const { // An OpenGL context is required there for the OpenGL calls like `glGetString`. // The format is QSurfaceFormat::defaultFormat() => OpenGL 3.3 defined in main.cpp. QOpenGLContext *context = new QOpenGLContext(); - context->create(); + if (!context->create()) + assert(false); QOpenGLFunctions *gl = context->functions(); // QOpenGLFunctions_3_3_Core cannot be initialized here on some systems like // macOS High Sierra and some Ubuntu environments. - + assert(gl); #ifdef _WIN32 const quint32 vendorId = WbSysInfo::gpuVendorId(gl); const quint32 rendererId = WbSysInfo::gpuDeviceId(gl); diff --git a/tests/api/controllers/supervisor_import_robot_node/supervisor_import_robot_node.c b/tests/api/controllers/supervisor_import_robot_node/supervisor_import_robot_node.c index 2553ab8f7cd..d19b237908e 100644 --- a/tests/api/controllers/supervisor_import_robot_node/supervisor_import_robot_node.c +++ b/tests/api/controllers/supervisor_import_robot_node/supervisor_import_robot_node.c @@ -25,7 +25,7 @@ int main(int argc, char **argv) { wb_robot_step(2 * TIME_STEP); // import robot object - FILE *fd = fopen("MyBot.txt", "rb"); + FILE *fd = fopen("MyBot.txt", "r"); char contents[4096]; const int n = fread(contents, 1, sizeof(contents), fd); fclose(fd); diff --git a/tests/test_suite.py b/tests/test_suite.py index dac4d398c09..705b45361c9 100755 --- a/tests/test_suite.py +++ b/tests/test_suite.py @@ -116,7 +116,7 @@ def setupWebots(): command = Command([webotsFullPath, '--sysinfo']) command.run() if command.returncode != 0: - raise RuntimeError('Error when getting the Webots information of the system') + raise RuntimeError('Error when getting the Webots information of the system: ' + command.output) webotsSysInfo = command.output.split('\n') return webotsFullPath, webotsVersion, webotsSysInfo, webotsEmptyWorldPath @@ -181,7 +181,7 @@ def generateWorldsList(groupName): # generate the list from the arguments if filesArguments: for file in filesArguments: - if f'/tests/{groupName}/' in file: + if (os.sep + 'tests' + os.sep + groupName + os.sep) in file: worldsList.append(file) # generate the list from 'ls worlds/*.wbt' @@ -201,13 +201,16 @@ def generateWorldsList(groupName): # speaker test not working on github action because of missing sound drivers # robot window and movie recording test not working on BETA Ubuntu 22.04 GitHub Action environment # billboard test not working in macos GitHub Action environment + # billboard and robot window not working on windows GitHub Action environment. if (not filename.endswith('_temp.wbt') and not ('GITHUB_ACTIONS' in os.environ and ( filename.endswith('speaker.wbt') or filename.endswith('local_proto_with_texture.wbt') or (filename.endswith('robot_window_html.wbt') and is_ubuntu_22_04) or (filename.endswith('supervisor_start_stop_movie.wbt') and is_ubuntu_22_04) or - (filename.endswith('billboard.wbt') and sys.platform == 'darwin') + (filename.endswith('billboard.wbt') and sys.platform == 'darwin') or + (filename.endswith('billboard.wbt') and sys.platform == 'win32') or + (filename.endswith('robot_window_html.wbt') and sys.platform == 'win32') ))): worldsList.append(filename) From 3c4a3c0abb0268d7ee90b86278052426d83c5b42 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Mon, 16 Sep 2024 12:50:09 -0700 Subject: [PATCH 17/19] Fix crash when using muscles and animatedGeometry in sequential worlds. (#6661) * Fix crash when using muscles and animatedGeometry in sequential worlds. The issue was that WbWrenVertexArrayFrameListener thought that the frame had already been handled because it had been handled in the previous world. * Add test case. --- src/webots/nodes/utils/WbWorld.cpp | 2 ++ .../utils/WbWrenVertexArrayFrameListener.cpp | 6 ++++- .../utils/WbWrenVertexArrayFrameListener.hpp | 1 + .../.gitignore | 1 + .../track_animated_geometry_muscles/Makefile | 25 +++++++++++++++++ .../track_animated_geometry_muscles.c | 21 +++++++++++++++ .../track_animated_geometry_muscles.wbt | 27 +++++++++++++++++++ 7 files changed, 82 insertions(+), 1 deletion(-) create mode 100644 tests/api/controllers/track_animated_geometry_muscles/.gitignore create mode 100644 tests/api/controllers/track_animated_geometry_muscles/Makefile create mode 100644 tests/api/controllers/track_animated_geometry_muscles/track_animated_geometry_muscles.c create mode 100644 tests/api/worlds/track_animated_geometry_muscles.wbt diff --git a/src/webots/nodes/utils/WbWorld.cpp b/src/webots/nodes/utils/WbWorld.cpp index 5f57f617ee2..d7f886c4234 100644 --- a/src/webots/nodes/utils/WbWorld.cpp +++ b/src/webots/nodes/utils/WbWorld.cpp @@ -57,6 +57,7 @@ #include "WbWorldInfo.hpp" #include "WbWrenOpenGlContext.hpp" #include "WbWrenRenderingContext.hpp" +#include "WbWrenVertexArrayFrameListener.hpp" #include "WbWriter.hpp" #include @@ -92,6 +93,7 @@ WbWorld::WbWorld(WbTokenizer *tokenizer) : mIsLoading(true), mIsCleaning(false), mIsVideoRecording(false) { + WbWrenVertexArrayFrameListener::resetLastUpdateTime(); gInstance = this; WbNode::setInstantiateMode(true); WbNode::setGlobalParentNode(NULL); diff --git a/src/webots/nodes/utils/WbWrenVertexArrayFrameListener.cpp b/src/webots/nodes/utils/WbWrenVertexArrayFrameListener.cpp index aa82509167a..81b64d69be8 100644 --- a/src/webots/nodes/utils/WbWrenVertexArrayFrameListener.cpp +++ b/src/webots/nodes/utils/WbWrenVertexArrayFrameListener.cpp @@ -21,9 +21,13 @@ #include WbWrenVertexArrayFrameListener *WbWrenVertexArrayFrameListener::cInstance = NULL; +static double lastUpdateTime = -1.0; + +void WbWrenVertexArrayFrameListener::resetLastUpdateTime() { + lastUpdateTime = -1.0; +} static void processEvent() { - static double lastUpdateTime = -1.0; const double currentTime = WbSimulationState::instance()->time(); if (currentTime == lastUpdateTime) return; diff --git a/src/webots/nodes/utils/WbWrenVertexArrayFrameListener.hpp b/src/webots/nodes/utils/WbWrenVertexArrayFrameListener.hpp index 420d454d6b8..d33d78d3b75 100644 --- a/src/webots/nodes/utils/WbWrenVertexArrayFrameListener.hpp +++ b/src/webots/nodes/utils/WbWrenVertexArrayFrameListener.hpp @@ -24,6 +24,7 @@ class WbWrenVertexArrayFrameListener { public: static WbWrenVertexArrayFrameListener *instance(); static void clear(); + static void resetLastUpdateTime(); void subscribeTrack(WbTrack *track); void unsubscribeTrack(WbTrack *track); diff --git a/tests/api/controllers/track_animated_geometry_muscles/.gitignore b/tests/api/controllers/track_animated_geometry_muscles/.gitignore new file mode 100644 index 00000000000..4d33fd88e8f --- /dev/null +++ b/tests/api/controllers/track_animated_geometry_muscles/.gitignore @@ -0,0 +1 @@ +/track_animated_geometry_muscles diff --git a/tests/api/controllers/track_animated_geometry_muscles/Makefile b/tests/api/controllers/track_animated_geometry_muscles/Makefile new file mode 100644 index 00000000000..137d83b4e39 --- /dev/null +++ b/tests/api/controllers/track_animated_geometry_muscles/Makefile @@ -0,0 +1,25 @@ +# Copyright 1996-2023 Cyberbotics Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Webots Makefile system +# +# You may add some variable definitions hereafter to customize the build process +# See documentation in $(WEBOTS_HOME_PATH)/resources/Makefile.include + + +# Do not modify the following: this includes Webots global Makefile.include +null := +space := $(null) $(null) +WEBOTS_HOME_PATH?=$(subst $(space),\ ,$(strip $(subst \,/,$(WEBOTS_HOME)))) +include $(WEBOTS_HOME_PATH)/resources/Makefile.include diff --git a/tests/api/controllers/track_animated_geometry_muscles/track_animated_geometry_muscles.c b/tests/api/controllers/track_animated_geometry_muscles/track_animated_geometry_muscles.c new file mode 100644 index 00000000000..c1dcc473fa4 --- /dev/null +++ b/tests/api/controllers/track_animated_geometry_muscles/track_animated_geometry_muscles.c @@ -0,0 +1,21 @@ +#include +#include +#include +#include +#include +#include + +#include "../../../lib/ts_assertion.h" +#include "../../../lib/ts_utils.h" + +#define TIME_STEP 32 + +int main(int argc, char **argv) { + ts_setup(argv[0]); + wb_robot_step(TIME_STEP); + // Test that a world with a Muscle node will load. See issue #6659. + wb_supervisor_world_load("../../worlds/supervisor_set_hinge_position_dynamic.wbt"); + + ts_send_success(); + return EXIT_SUCCESS; +} diff --git a/tests/api/worlds/track_animated_geometry_muscles.wbt b/tests/api/worlds/track_animated_geometry_muscles.wbt new file mode 100644 index 00000000000..37c309f770d --- /dev/null +++ b/tests/api/worlds/track_animated_geometry_muscles.wbt @@ -0,0 +1,27 @@ +#VRML_SIM R2024a utf8 + +EXTERNPROTO "webots://tests/default/protos/TestSuiteEmitter.proto" +EXTERNPROTO "webots://tests/default/protos/TestSuiteSupervisor.proto" + +WorldInfo { +} +Viewpoint { +} +Background { +} +Robot { + children [ + Track { + children [ + TrackWheel { + } + ] + animatedGeometry Shape { + } + } + ] + controller "track_animated_geometry_muscles" + supervisor TRUE +} +TestSuiteSupervisor { +} From 8977e52d53827ecf5e2d6097d790e44d357720f0 Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Sat, 21 Sep 2024 06:40:11 -0700 Subject: [PATCH 18/19] Ensure that test_suite.pt only tries to build controllers which have Makefiles. (#6666) Switching between master and develop branches can result in controller directories sticking around that only contain the results of a build on the previous branch. test_suite.py was trying (and failing) to run make in those directories. --- tests/api/controllers/Makefile | 2 +- tests/cache/controllers/Makefile | 2 +- tests/other_api/controllers/Makefile | 2 +- tests/parser/controllers/Makefile | 2 +- tests/physics/controllers/Makefile | 2 +- tests/protos/controllers/Makefile | 2 +- tests/rendering/controllers/Makefile | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/api/controllers/Makefile b/tests/api/controllers/Makefile index 3149288896d..b0fec7529fb 100644 --- a/tests/api/controllers/Makefile +++ b/tests/api/controllers/Makefile @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -DIRECTORIES=$(subst test_suite_supervisor,,$(subst Makefile,,$(shell ls))) +DIRECTORIES=$(dir $(wildcard */Makefile)) TARGETS = $(DIRECTORIES:=.Makefile) .PHONY: release debug profile clean diff --git a/tests/cache/controllers/Makefile b/tests/cache/controllers/Makefile index 3149288896d..b0fec7529fb 100644 --- a/tests/cache/controllers/Makefile +++ b/tests/cache/controllers/Makefile @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -DIRECTORIES=$(subst test_suite_supervisor,,$(subst Makefile,,$(shell ls))) +DIRECTORIES=$(dir $(wildcard */Makefile)) TARGETS = $(DIRECTORIES:=.Makefile) .PHONY: release debug profile clean diff --git a/tests/other_api/controllers/Makefile b/tests/other_api/controllers/Makefile index 3149288896d..b0fec7529fb 100644 --- a/tests/other_api/controllers/Makefile +++ b/tests/other_api/controllers/Makefile @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -DIRECTORIES=$(subst test_suite_supervisor,,$(subst Makefile,,$(shell ls))) +DIRECTORIES=$(dir $(wildcard */Makefile)) TARGETS = $(DIRECTORIES:=.Makefile) .PHONY: release debug profile clean diff --git a/tests/parser/controllers/Makefile b/tests/parser/controllers/Makefile index 3149288896d..b0fec7529fb 100644 --- a/tests/parser/controllers/Makefile +++ b/tests/parser/controllers/Makefile @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -DIRECTORIES=$(subst test_suite_supervisor,,$(subst Makefile,,$(shell ls))) +DIRECTORIES=$(dir $(wildcard */Makefile)) TARGETS = $(DIRECTORIES:=.Makefile) .PHONY: release debug profile clean diff --git a/tests/physics/controllers/Makefile b/tests/physics/controllers/Makefile index 3149288896d..b0fec7529fb 100644 --- a/tests/physics/controllers/Makefile +++ b/tests/physics/controllers/Makefile @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -DIRECTORIES=$(subst test_suite_supervisor,,$(subst Makefile,,$(shell ls))) +DIRECTORIES=$(dir $(wildcard */Makefile)) TARGETS = $(DIRECTORIES:=.Makefile) .PHONY: release debug profile clean diff --git a/tests/protos/controllers/Makefile b/tests/protos/controllers/Makefile index 3149288896d..b0fec7529fb 100644 --- a/tests/protos/controllers/Makefile +++ b/tests/protos/controllers/Makefile @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -DIRECTORIES=$(subst test_suite_supervisor,,$(subst Makefile,,$(shell ls))) +DIRECTORIES=$(dir $(wildcard */Makefile)) TARGETS = $(DIRECTORIES:=.Makefile) .PHONY: release debug profile clean diff --git a/tests/rendering/controllers/Makefile b/tests/rendering/controllers/Makefile index 3149288896d..b0fec7529fb 100644 --- a/tests/rendering/controllers/Makefile +++ b/tests/rendering/controllers/Makefile @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -DIRECTORIES=$(subst test_suite_supervisor,,$(subst Makefile,,$(shell ls))) +DIRECTORIES=$(dir $(wildcard */Makefile)) TARGETS = $(DIRECTORIES:=.Makefile) .PHONY: release debug profile clean From 0a1f1eb4ce7862330620c3a80fed253adaf05adf Mon Sep 17 00:00:00 2001 From: Dean Brettle Date: Sat, 21 Sep 2024 07:32:09 -0700 Subject: [PATCH 19/19] Fix-references-to-raw-github-content (#6667) * Replace reference to raw github content with webots:// in ShapeWithFixedWebTexture.proto. The reference to a file in the github develop branch was causing that file to be left as changed when running test_suite.py in the develop branch. --- tests/cache/protos/ShapeWithFixedWebTexture.proto | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/cache/protos/ShapeWithFixedWebTexture.proto b/tests/cache/protos/ShapeWithFixedWebTexture.proto index 12ab7a375d2..2b8fcdeac32 100644 --- a/tests/cache/protos/ShapeWithFixedWebTexture.proto +++ b/tests/cache/protos/ShapeWithFixedWebTexture.proto @@ -8,7 +8,7 @@ PROTO ShapeWithFixedWebTexture [ appearance PBRAppearance { baseColorMap ImageTexture { url [ - "https://raw.githubusercontent.com/cyberbotics/webots/develop/tests/cache/protos/textures/blue_texture.jpg" + "webots://tests/cache/protos/textures/blue_texture.jpg" ] } }