Skip to content

Commit

Permalink
Allow Indirect Field Accesses in Proto Templates (#6614)
Browse files Browse the repository at this point in the history
* allow indirectly referencing proto fields

* update other regular expression

* add test

* fix test worlds

* update changelog

* use spaces for indentation

* update proto headers

* run clang-format

* enable fields variable with a tag

* update regex tests

* update docs

* update changelog

* run clang-format

* add curly brackets

* fix comment indentation

* fix proto name

* fix controller name

* update the changelog

* fix indentation

Co-authored-by: Olivier Michel <Olivier.Michel@cyberbotics.com>

---------

Co-authored-by: Olivier Michel <Olivier.Michel@cyberbotics.com>
  • Loading branch information
CoolSpy3 and omichel committed Aug 12, 2024
1 parent 6b49409 commit fe2879c
Show file tree
Hide file tree
Showing 12 changed files with 213 additions and 27 deletions.
1 change: 1 addition & 0 deletions docs/reference/changelog-r2024.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Released on December **th, 2023.
- New Features
- **Change the name of the web scene format from `X3D` to `W3D` ([#6280](https://github.com/cyberbotics/webots/pull/6280)).**
- Removed support for macOS 11 "Big Sur" and added support for macOS 14 "Sonoma" ([#6580](https://github.com/cyberbotics/webots/pull/6580)).
- Added the `indirectFieldAccess` tag to allow the `fields` variable to be used in proto templates without referencing a specific field ([#6614](https://github.com/cyberbotics/webots/pull/6614)).
- Enhancements
- Improved the image range of the rotating [Lidar](lidar.md) ([#6324](https://github.com/cyberbotics/webots/pull/6324)).
- Cleanup
Expand Down
10 changes: 10 additions & 0 deletions docs/reference/javascript-procedural-proto.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ The first represents the effective value of the field (for instance the one defi
- As shown in [this table](#vrml97-type-to-javascript-type-conversion), the conversion of a VRML97 node is an object.
This object contains the following keys: "node\_name" containing the VRML97 node name and "fields" which is in turn an object containing the JavaScript representation of the VRML97 node fields.
This object is equal to `undefined` if the VRML97 node is not defined (`NULL`).
- By default, the parser only detects fields that are accessed directly in the template statements (i.e. `fields.appearance`).
If you would like to access the `fields` object without referencing a specific field, you can add the following line at the beginning of the PROTO file: `# tags: indirectFieldAccess`.
- Objects that are part of [ECMA-262](http://www.ecma-international.org/publications/standards/Ecma-262.htm) are built-in and globally accessible, such as `Math`, `Date` and `String`.
- The `context` object provides contextual information about the PROTO.
Table [this table](#content-of-the-context-object) shows the available information and the corresponding keys.
Expand Down Expand Up @@ -823,6 +825,14 @@ The location of this path can be retrieved from the `context` field object, see

%end

### PROTO Regeneration
When a field used in a template statement is modified, the PROTO node is regenerated.
This means that the template statements are re-evaluated and the PROTO node is reloaded in the world.
For most nodes, this behavior should not affect the simulation.
However, special care should be taken when using PROTOs that have side effects (e.g. writing to a file).
Additionally, Robot nodes will restart their controllers when regenerated.
Using `tags: indirectFieldAccess` in the PROTO file will cause the PROTO to be regenerated whenever any field is modified, even if it is not used in a template statement.

### Optimization

By default, PROTO files are considered to be deterministic.
Expand Down
10 changes: 10 additions & 0 deletions docs/reference/lua-procedural-proto.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ The conversion between the VRML97 types and the Lua types is detailed in [this t
This dictionary contains the following keys: "node\_name" containing the VRML97 node name and "fields" which is a dictionary containing the Lua representation of the VRML97 node fields.
This dictionary is equal to `nil` if the VRML97 node is not defined (`NULL`).
For example, in the SimpleStairs example below, the `fields.appearance.node_name` key contains the `'Appearance'` string.
- By default, the parser only detects fields that are accessed directly in the template statements (i.e. `fields.appearance`).
If you would like to access the `fields` object without referencing a specific field, you can add the following line at the beginning of the PROTO file: `# tags: indirectFieldAccess`.
- The `context` dictionary provides contextual information about the PROTO.
Table [this table](#content-of-the-context-dictionary) shows the available information and its corresponding keys.
- The VRML97 comment ("#") prevails over the Lua statements.
Expand Down Expand Up @@ -88,6 +90,14 @@ The following standard fonts are available to write on the texture:

In addition to these fonts, it is possible to add other TrueType fonts file in your `PROJECT_HOME/fonts` directory.

### PROTO Regeneration
When a field used in a template statement is modified, the PROTO node is regenerated.
This means that the template statements are re-evaluated and the PROTO node is reloaded in the world.
For most nodes, this behavior should not affect the simulation.
However, special care should be taken when using PROTOs that have side effects (e.g. writing to a file).
Additionally, Robot nodes will restart their controllers when regenerated.
Using `tags: indirectFieldAccess` in the PROTO file will cause the PROTO to be regenerated whenever any field is modified, even if it is not used in a template statement.

### Optimization

By default, PROTO files are considered to be deterministic.
Expand Down
64 changes: 38 additions & 26 deletions src/webots/vrml/WbProtoModel.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ WbProtoModel::WbProtoModel(WbTokenizer *tokenizer, const QString &worldPath, con
mDocumentationUrl = tokenizer->documentationUrl();
mTemplateLanguage = tokenizer->templateLanguage();
mIsDeterministic = !mTags.contains("nonDeterministic");
mHasIndirectFieldAccess = mTags.contains("indirectFieldAccess");

WbParser parser(tokenizer);
while (tokenizer->peekWord() == "EXTERNPROTO" || tokenizer->peekWord() == "IMPORTABLE") // consume EXTERNPROTO declarations
Expand Down Expand Up @@ -209,15 +210,23 @@ WbProtoModel::WbProtoModel(WbTokenizer *tokenizer, const QString &worldPath, con
previousToken = token;
token = tokenizer->nextToken();

if (mHasIndirectFieldAccess) {
foreach (WbFieldModel *model, mFieldModels)
model->setTemplateRegenerator(true);
}

if (token->isTemplateStatement()) {
mTemplate = true;

foreach (WbFieldModel *model, mFieldModels) {
// condition explanation: if (token contains modelName and not a Lua identifier containing modelName such as
// "my_awesome_modelName")
if (token->word().contains(QRegularExpression(
QString("(^|[^a-zA-Z0-9_])fields\\.%1($|[^a-zA-Z0-9_])").arg(QRegularExpression::escape(model->name()))))) {
model->setTemplateRegenerator(true);
if (!mHasIndirectFieldAccess) { // If the proto has indirect field access, we've already set the fields as template
// regenerators
foreach (WbFieldModel *model, mFieldModels) {
// condition explanation: if (token contains modelName and not a Lua identifier containing modelName such as
// "my_awesome_modelName") or (token contains fields and not a Lua identifier containing fields such as "my_fields")
if (token->word().contains(
QRegularExpression(QString("(^|\\W)fields\\.%1($|\\W)").arg(QRegularExpression::escape(model->name()))))) {
model->setTemplateRegenerator(true);
}
}
}
} else if (readBaseType) {
Expand Down Expand Up @@ -274,26 +283,29 @@ WbProtoModel::WbProtoModel(WbTokenizer *tokenizer, const QString &worldPath, con
throw 0;
}
} else if (token->isString()) {
// check which parameter need to regenerate the template instance from inside a string
foreach (WbFieldModel *model, mFieldModels) {
// regex test cases:
// "You know nothing, John Snow." => false
// "%{=fields.model->name()}%" => false
// "%{= fields.model->name().value.x }% %{= fields.model->name().value.y }%" => true
// "abc %{= fields.model->name().value.y }% def" => true
// "%{= 17 % fields.model->name().value.y * 88 }%" => true
// "fields.model->name().value.y" => false
// "%{}% fields.model->name().value.y %{}%" => false
// "%{ a = \"fields.model->name().value.y\" }%" => false
// "%{= \"fields.model->name().value.y\" }%" => false
// "%{= fields.model->name().value.y }%" => true
if (token->word().contains(QRegularExpression(QString("%1(?:(?!%2|\").)*fields\\.%3(?:(?!%4|\").)*%5")
.arg(open)
.arg(close)
.arg(QRegularExpression::escape(model->name()))
.arg(close)
.arg(close))))
model->setTemplateRegenerator(true);
if (!mHasIndirectFieldAccess) { // If the proto has indirect field access, we've already set the fields as template
// regenerators
// check which parameter need to regenerate the template instance from inside a string
foreach (WbFieldModel *model, mFieldModels) {
// regex test cases:
// "You know nothing, John Snow." => false
// "%{=fields.model->name()}%" => true
// "%{= fields.model->name().value.x }% %{= fields.model->name().value.y }%" => true
// "abc %{= fields.model->name().value.y }% def" => true
// "%{= 17 % fields.model->name().value.y * 88 }%" => true
// "fields.model->name().value.y" => false
// "%{}% fields.model->name().value.y %{}%" => false
// "%{ a = \"fields.model->name().value.y\" }%" => false
// "%{= \"fields.model->name().value.y\" }%" => false
// "%{= fields.model->name().value.y }%" => true
if (token->word().contains(QRegularExpression(QString("%1(?:(?!%2|\").)*fields\\.%3(?:(?!%4|\").)*%5")
.arg(open)
.arg(close)
.arg(QRegularExpression::escape(model->name()))
.arg(close)
.arg(close))))
model->setTemplateRegenerator(true);
}
}
}

Expand Down
3 changes: 2 additions & 1 deletion src/webots/vrml/WbProtoModel.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,8 @@ class WbProtoModel : public QObject {
WbVersion mFileVersion;
QString mName;
QString mInfo;
bool mIsDeterministic; // i.e doesn't have the 'nonDeterministic' tag
bool mIsDeterministic; // i.e doesn't have the 'nonDeterministic' tag
bool mHasIndirectFieldAccess; // i.e. has the 'indirectFieldAccess' tag
QList<WbFieldModel *> mFieldModels;

QString mUrl; // how the PROTO is referenced
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/template_indirect_field_access
19 changes: 19 additions & 0 deletions tests/protos/controllers/template_indirect_field_access/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Copyright 1996-2023 Cyberbotics Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

### Do not modify: this includes Webots global Makefile.include
null :=
space := $(null) $(null)
WEBOTS_HOME_PATH?=$(subst $(space),\ ,$(strip $(subst \,/,$(WEBOTS_HOME))))
include $(WEBOTS_HOME_PATH)/resources/Makefile.include
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#include <webots/motor.h>
#include <webots/robot.h>
#include <webots/supervisor.h>

#include "../../../lib/ts_assertion.h"
#include "../../../lib/ts_utils.h"

#define TIME_STEP 32

int main(int argc, char **argv) {
ts_setup(argv[1]); // give the controller args
WbNodeRef proto = wb_supervisor_node_get_from_def("PROTO_template_indirect_field_access");
WbFieldRef radarCrossSection = wb_supervisor_node_get_proto_field(proto, "radarCrossSection");
ts_assert_double_in_delta(wb_supervisor_field_get_sf_float(radarCrossSection), 6.0, 0.0001,
"radarCrossSection should be 6.0");
ts_send_success();

return EXIT_SUCCESS;
}
18 changes: 18 additions & 0 deletions tests/protos/protos/TemplateIndirectFieldAccess.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#VRML_SIM R2024a utf8
# tags: indirectFieldAccess
# template language: javascript

PROTO TemplateIndirectFieldAccess [
unconnectedField SFFloat field1 1
unconnectedField SFFloat field2 5
]
{
Solid {
%<
function sum(fieldsObj) {
return fieldsObj.field1.value + fieldsObj.field2.value;
}
>%
radarCrossSection %<= sum(fields) >%
}
}
17 changes: 17 additions & 0 deletions tests/protos/protos/TemplateIndirectFieldAccessLua.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#VRML_SIM R2024a utf8
# tags: indirectFieldAccess

PROTO TemplateIndirectFieldAccessLua [
unconnectedField SFFloat field1 1
unconnectedField SFFloat field2 5
]
{
Solid {
%{
function sum(fieldsObj)
return fieldsObj.field1.value + fieldsObj.field2.value;
end
}%
radarCrossSection %{= sum(fields) }%
}
}
39 changes: 39 additions & 0 deletions tests/protos/worlds/template_indirect_field_access.wbt
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#VRML_SIM R2024a utf8

EXTERNPROTO "webots://tests/protos/protos/TemplateIndirectFieldAccess.proto"
EXTERNPROTO "webots://tests/default/protos/TestSuiteEmitter.proto"
EXTERNPROTO "webots://tests/default/protos/TestSuiteSupervisor.proto"

WorldInfo {
coordinateSystem "NUE"
lineScale 1
}
Viewpoint {
orientation -0.8999515417256047 0.42450878383113183 0.0993957493855782 0.509061
position 0.303563 0.46348 0.936228
}
Background {
skyColor [
0 0.811765 0.992157
]
}
DirectionalLight {
direction 0 0.5 -1
}
Robot {
translation 0.1 0 0.15
rotation 0 1 0 1.5708
children [
DEF PROTO_template_indirect_field_access TemplateIndirectFieldAccess {
}
TestSuiteEmitter {
}
]
controller "template_indirect_field_access"
controllerArgs [
"template_indirect_field_access"
]
supervisor TRUE
}
TestSuiteSupervisor {
}
39 changes: 39 additions & 0 deletions tests/protos/worlds/template_indirect_field_access_lua.wbt
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#VRML_SIM R2024a utf8

EXTERNPROTO "webots://tests/protos/protos/TemplateIndirectFieldAccessLua.proto"
EXTERNPROTO "webots://tests/default/protos/TestSuiteEmitter.proto"
EXTERNPROTO "webots://tests/default/protos/TestSuiteSupervisor.proto"

WorldInfo {
coordinateSystem "NUE"
lineScale 1
}
Viewpoint {
orientation -0.8999515417256047 0.42450878383113183 0.0993957493855782 0.509061
position 0.303563 0.46348 0.936228
}
Background {
skyColor [
0 0.811765 0.992157
]
}
DirectionalLight {
direction 0 0.5 -1
}
Robot {
translation 0.1 0 0.15
rotation 0 1 0 1.5708
children [
DEF PROTO_template_indirect_field_access TemplateIndirectFieldAccessLua {
}
TestSuiteEmitter {
}
]
controller "template_indirect_field_access"
controllerArgs [
"template_indirect_field_access (lua)"
]
supervisor TRUE
}
TestSuiteSupervisor {
}

0 comments on commit fe2879c

Please sign in to comment.