PROCEED MS JS Editor
The IDE sits within the ProcessBpmnEditor component, since editing script tasks is part of editing a BPMN process. When a script task is selected within the BPMN editor, the IDE can be opened to edit the script task’s code.
Most of the Proceed IDE lives within files in its own, newly-created folder within the components section of the source folder src. There is, however, another script-editor-helpers.js file in the helpers folder. A few methods regarding BPMN parsing were added to the bpmn-editor-helpers.js file in the same folder. Lastly, the ProcessBpmnEditor component includes the ScriptingIde component.
The Proceed IDE is fully integrated with the Proceed Management System’s Vue components and also provides its own Vue components. It reads from and commits to the project’s VueX stores and makes minor modifications to the processStore and the processEditorStore. It mainly interacts with the BPMN editor’s processEditorStore. All of the markup consists of Vuetify components, re-creating the same look and feel as the rest of the Proceed Management System.
The entire Proceed IDE is written within a single component which itself is structured in sub-components, as depicted below:
The main component is called ScriptingIde and is mounted within the existing BpmnProcessEditor component. It renders the list of available processes and script tasks in the left sidebar and includes the ScriptEditor component. The ScriptEditor component handles the state for the currently open script task, analyzes code, generates suggestions, messages and more.
It also includes the MonacoEditor in the main column and the three tab components in the tabbed sidebar on the right. The MonacoEditor component takes care of implementing the Monaco library and rendering the editor field. It acts as an intermediary between Proceed and the editor’s APIs. The tab components CapabilitiesTab, DevicesTab and VariablesTab provide various tools to help the programmer develop script tasks for the Proceed system.
The IDE must know all the script tasks which exist in the system in order to do both -- show them in the script task tree and show their code in the code editor. However, the one place in the VueX-based system which contains data about all script tasks at once, the ProcessStore, isn’t up-to-date when a process is being edited within the ProcessBpmnEditor component.
The ProcessBpmnEditor component duplicates the currently edited process into the ProcessEditorStore and only commits all changes back to the main ProcessStore when the user saves the entire process. The benefit of this approach is only updating the ProcessStore when the user decides they are done with editing the process, and not potentially saving a broken process in between, which could lead to a faulty process being dispatched. At the same time, this means that additional, renamed or removed script tasks are not reflected in the main ProcessStore while the process is unsaved.
This concludes that the Proceed IDE’s menu tree, which should consist of all script tasks in the system, must come from two sources. Thus, the main ScriptingIde component of the Proceed IDE has two computed properties:
- processElements, which returns a list of all script tasks of the currently open process, computed from the processes XML from the currently worked on ProcessEditorStore, and
- otherElements, which returns a list of all script tasks from all other processes, computed from those processes XML documents.
The computed property menuTree then combines both, while all script tasks which don’t belong to the currently open process are marked read-only in the fashion of only committing to the ProcessEditorStore, which only stores the currently open process. Each menuTree item holds a reference to the original XML element, which includes the script data which is being passed down as the value property to all child components.
Why was Monaco selected as the underlying IDE?
While writing the Proceed IDE, there was no need to reinvent the wheel when it comes to the actual code editor field. There are plenty of libraries out there which can be used to replace the previously existing textarea field to write code.
To choose the best suited script editor, a few criteria were defined:
- web-based: the editor has to use only web frontend technologies so it can run within the Proceed environment,
- language features: the editor comes with all common javascript language formatting support like indentation, syntax check and colored syntax highlighting,
- open source: the editor has a license which allows free commercial usage within Proceed, ideally fully open source
- documented and maintained: the editor and its APIs are well-documented and there are regular updates to the software,
- autocomplete: the editor preferably comes with a ready interface to provide suggestions, which the editor will display while writing code,
- design: the editor must have a modern look-and-feel, be intuitive and must not be clunky.
From a comprehensive list of available web-based code editors, 5 were considered which were up-to-date, meaning they had a release within the last two years, and their main goal is to provide a solid, integrable frontend editor only, not a web-based IDE clone which includes server-side components and other UI elements.
For this reason, the Orion project by the Eclipse Foundation, the CodeAnywhere editor, Eclipse Che as well as the Atom editor were not considered. The remaining five editors were investigated to see if they meet the criteria defined above.
As seen in in the table, most editors are licensed under the MIT licence which grants free commercial usage as well as the ability to modify, distribute and sublicense the software[24] which comes in handy within the PROCEED project.
The Ace editor in turn uses a custom license, which may also be suitable for the Proceed project, however it would be required to hire a lawyer to confirm, which was not part of this thesis and thus the Ace editor failed the license criterium. Most editors are also frequently being updated and have an up-to-date modern, simple and elegant UI. The only exception to this rule is the MarkItUp editor whose last release was early 2018 and it looks and feels old and dated. Thus, the MarkItUp editor failed both criteria. All editors investigated came with built-in syntax highlighting for JavaScript and automated indentation help. However, both CodeFlask and MarkItUp don’t highlight syntax errors and thus fail the language feature criteria.
When it comes to documentation, it’s not straightforward to define objective criteria. However, all editors investigated had public documentation and guides to get started, which is why they all passed the documentation criteria. At this point, both the CodeMirror editor and Monaco editor fulfill all criteria. The decision between these two editors came down to their auto completion feature. Both editors are the only ones with built-in help for providing the programmer with on-the-fly suggestions. The auto completion API in Monaco Editor is much better documented, with more examples readily available, and it is easy and intuitive to extend. Thus, the Monaco Editor was chosen as the code editor for the Proceed IDE.
Integrating the Monaco Editor into the existing Proceed Management system requires two steps. Firstly, a custom Vue component has to be created, and secondly, Monaco’s JavaScript workers must be enabled.
The Monaco Editor is integrated into the Proceed Management System using a custom Vue component called MonacoEditor.
It allows to pass a value property for the code value, as well as a an array of suggestions to autocomplete when the programmer writes code.
Furthermore, an array of message can be passed to the MonacoEditor component together with a line number where they should be displayed either as a warning or an error (type).
A readonly property, which is false by default, allows to turn off writing in the editor.
Last but not least, the reloadFlag property allows to pass an identification string for the current version of the value passed to the component.
Whenever the reloadFlag changes, the MonacoEditor reloads the value property into the editor.
This is not being done automatically because of a vicious update circle if the value is being updated automatically.
Whenever a user modifies the content of the editor, an event is triggered upwards up to the ScriptingIde component, which updates the value of the selected element.
Immediately, the same value is being propagated down again via the properties until it reaches the MonacoEditor component.
There, if the content of the Monaco editor was being updated immediately upon the change (since the value property within the MonacoEditor component still holds the value before the change was made in the editor), it would reload in an interrupting way to the user: the editor would freeze for a few milliseconds and the cursor would be reset to the first position.
This means that in edge cases, typed letters could potentially get lost and users are interrupted in their writing flow.
Thus, the compromise was made to require any parent component which includes the MonacoEditor to change the reloadFlag property whenever an actual reload of the Monaco editor is supposed to happen.
Within the Proceed IDE, this is simply done by mapping the reloadFlag property of the MonacoEditor component to the id attribute of the currently open element.
This way, whenever a different script task is opened, the editor is being forced to reload its content, but not
while the user is typing.
Since the Monaco editor makes use of extensive libraries interpreting the entire code on every change, its performance relies heavily on a JavaScript worker running in the background. Workers are separate threads which can execute tasks in parallel to the main application. Workers execute their own JavaScript file.
Since the Proceed Management System uses Webpack to bundle all source files into one entry file, the Webpack configuration file had to be adjusted to create a separate entry point for Monaco’s JavaScript language worker. Monaco even provides a webpack plugin which can simply be included in the webpack configuration file to provide the entry point needed for the language worker.
const MonacoWebpackPlugin = require('monaco-editor-webpack-plugin');
let config = {
...
plugins: [
...
new MonacoWebpackPlugin({
languages: ['javascript', 'typescript'],
})
],
...
} In order to help the programmer select and describe the capabilities needed, firstly they need to know which capabilities are available, and which parameters are needed.
A tab in the right sidebar shows all devices available in the network, together with their capabilities and parameters. To get a very detailed view about each capability, a click on an info button opens up a detailed documentation about each capability, including information such as name, schema URI, return value and concrete descriptions about each parameter, including data types and advanced validation rules. The overlay also shows a sample implementation code of the capability which can be copied for own usage. A click on any capability pastes the sample code into the code editor.
TODO: At the time of writing this thesis, the data about the capabilities was not fully available from the internal store yet. Thus, the decision was made to mock the data about available capabilities. This means, that the data in form of JavaScript arrays and objects is currently hard-coded in a separate JavaScript file and embedded within the DevicesTab component.
To further support the programmer in writing code, both for managing required capabilities and parameters as well as warning a programmer when the capability they require cannot be executed on any device in the network, the Proceed IDE must know which capabilities the programmer calls in their code.
A sample capability call looks as follows:
services.system().startCapability('PhotographAction', { resolution: 720 });The code above starts the capability PhotographAction and passes a resolution parameter. Given that the programmer uses the capability call above, the Proceed IDE must know that the programmer needs a capability with the given name and parameters to execute their code, and for that, their code must be analyzed on the fly while they are writing.
There are two ways of parsing the programmer’s code.
Regular Expressions
It is possible to parse the entire user code using regular expressions and find out which capabilities are called and parameters required. For simplicity’s sake, the regular expression was split into two parts: the first function searches the capability calls and understands which capabilities are required, and then passes the entire parameter part to a second regular expression which returns a list of parameters used.
ATTENTION: small errors in the table (missing characters) because of Latex unescaped characters
The regular expression shown in Figure above is used to find capability calls within the programmer’s code. The parameter string captured as part of a group is then sent to the next regular expression which finds out what parameters are being used.
Both regular expressions are called in a loop until the last result is found within the search string.
The regex approach is simple, yet it has some flaws and fails to deliver optimal results in certain edge cases, and even quite common cases. For example, the regular expressions above would find dimensions, width and height as parameters in the following piece of code:
services.system().capability().startCapability('PhotographAction', {
dimensions: {
width: 100,
height: 200
}
});That is not correct, though - “width” and “height” are part of the value of the “dimensions” parameter, not parameters themselves! The regex does not differentiate, it will instead think of “width” and “height” as first-level parameters. Likewise, the following code snippet is not parsed correctly:
services.system().capability().startCapability('PhotographAction', {
items: items.each((e) => { return calculateLocation( { coords: e.coords }); }),
resolution: 1080
});The }); closing out the function called within the "items" parameter tricks the regex into believing the capability call is finished, and the “resolution” parameter is missed.
Instead, the regex believes that the “coords” attribute is a parameter of the “PhotographAction” and returns
that instead, even though it is part of the “each” loop over the “items” array.
Thus, the regex implementation is simple and effective but far from perfect.
Esprima Library
An alternative is the esprima library which provides a complex algorithm to parse JavaScript code. It returns a JavaScript tree of objects which represents and labels the entire code. It is possible to recursively investigate the entire tree and find capability function calls. Since it is a nested tree, the problem with the simple regex solution described above does not exist. Nested elements are displayed as nested elements within the resulting tree.
The final implementation of the Proceed IDE for this project, however, does not make use of the esprima library. That's because there can be many different labels in the tree, which can get very complicated in large and nested code. It is unlikely that within reasonable time, a solution could be implemented which would capture all edge cases and correctly iterate over the entire tree to find all capabilities.
Restrictions
Unfortunately both approaches only parse the given code and can not find capabilities or parameters, when parameters or capability names are
- declared outside of the capability call in a variable,
- computed dynamically by a function, or
- passed into the script by a process variable.
These cases require execution of the code and keeping an active heap of used variables instead of parsing it. This is impossible, since the state of the given process variables and results of capability calls or HTTP calls and other environment variables is unknown, can change and lead to different results.
The previous section described how it is possible to automatically detect many capability calls and their parameters made by the programmer, but not all of them.
The Proceed system, however, requires that a script task includes a definition of all capabilities necessary and their required parameters. Since the Proceed IDE is not able to find out correctly all capabilities automatically, the programmer is required to help. There are several ways a programmer can specify the needed capabilities and parameters.
Require Statements
It is common in the JavaScript world to point out needed scripts and libraries using the require statement, which usually tells a bundler to include certain references and bundle certain parts of code.
As part of Proceed, a require statement could be used to define needed capabilities and their parameters. This could look as follows:
require('PhotographAction', ['resolution']);This would tell the Proceed IDE that the programmer needs a PhotographAction capability which accepts a resolution parameter - it is required.
In order to prevent the same problems as with parsing the capability calls themselves, a linting rule for the code would say that “require” statements must not be dynamic and their parameters must be plain text, and also that “require” statements must be on top of the script.
The downside of this approach is that while the “require” statements are written in the code, they can not actually be stored with the code, since the engines the code will run on do not know how to interpret these “require” statements. They don’t actually belong to the code. They would have to be parsed and removed, and their requirements stored as XML within the BPMN diagram. But if the script task is opened in the Proceed IDE again, they would have to be re-added to the top of the script.
This is not a hard thing to do, but the decision was made that this is unnecessary overhead and cumbersome for a programmer to maintain, also error-prone if any “require” statement fails to parse.
Capabilities Inspector Tab
Another way to let the developer explicitly define required capabilities and their required parameters is another tab in the sidebar, which lets the developer freely define their requirements. The following figure shows a tab where the programmer can freely define their required capabilities and parameters.
Also, capability parameter requirements, which were detected in the code, appear as suggestions below the capability list and can be added to the requirements with a single click. This approach gives the developer the most freedom to manually adjust and define any requirements, while making easy one-click suggestions for all capability-parameter requirements detected automatically.
Mapping to Devices in the Network
While the devices tab in the right sidebar shows all available network devices and their capabilities together with their requirements, it also highlights any capabilities that could potentially run capabilities mentioned in the code. This gives the developer quick real-time feedback if the required capabilities exist in the network or not.
However, this can not assure the developer that the exact same device will run the capability at runtime in a created process instance, since that is decided by the process engine at runtime. Thus, the decision was made to merely showcase which devices are potentially available for the capabilities called in the code.
Highlighting Capabilities
To indicate which capabilities are used in the code and to revise the automatic detection (if it is wrong), the capabilities are highlighted. This follows certain rules.
A capability from a certain device is highlighted, when at least one capability call in the code is found which fulfills the requirements. This is the case when all its required parameters are passed in a given capability call. Additional parameters passed to the capability call but not required by the device’s capability description will be ignored.
For example, a capability call in the code for capability PhotographAction passing parameters resolution and direction will highlight the capability PhotographAction by device "A" which requires the “resolution” parameter, but it will not highlight the PhotographAction by device "B" which requires a “width” parameter.
When mapping capabilities, it does not matter whether the capability call uses the capability shortname like PhotographAction or the schema URI defined in the capability definition, like “https://schema.org/PhotographAction”. Both fields are being checked against the capability call.
TODO: use own Capability module from the Engine to detect this in the source code.
The programmer must be able to easily reuse parts of the code previously written somewhere else.
The first and foremost possibility of the programmer hence must be to view any other code written in the system. Thus, the entire menuTree of all script tasks available in all processes in the Management System is being displayed in the left column. The script tasks are grouped by process, whereas the process which is currently open in the BPMN editor is put on top.
Clicking on any script task opens its code in the MonacoEditor component and loads its capability mapping in the Capabilities tab which starts interpreting the code looking for further capabilities. Similarly, the Devices tab refreshes the capability highlights according to the capabilities detected in the code.
Before the code from a given script task is passed to the MonacoEditor, it is separated from the library code within the code bundle, which is described in detail in the “Bundling Code and Library” section later in this thesis.
All script tasks can be opened in parallel without saving. They are being saved automatically when the Proceed IDE is being closed. As stated previously, all script tasks from other processes than the one currently opened in the BPMN editor are opened in read-only mode, meaning they can be read and copied, but writing is disabled.
Once a programmer reuses code several times, they might want to not copy and paste the same code over and over again, but rather define it once and reference it in the form of a function call somewhere else. To support this, some sort of library is needed where globally used functions can be defined.
The author of this thesis decided not to implement a global script library for all processes, the reason for this being that all processes are stored in their respective XML document and may be independently transported to other management systems. Those management systems must allow displaying and editing of that processes library versus another processes library which may have come from another management system. Thus, one single library cannot exist, and the decision was made to provide one independent library per process.
This means, that if a programmer wants to reuse a function within multiple script tasks within a process, they can define it in the processes library and reference it in any script task they write. If, however, they want to reference this function within a script task of another process, they must copy the function to that processes library first.
TODO: one global library for one MS. Parsing out the lib sections of every script and extending the global lib of one MS.
The actual implementation of storing and using the library is described as part of the following chapter.
Until this section, the text describes how script task code is being read, displayed and how a programmer may manipulate the code. This chapter now goes on to explain how the modified code and other data is permanently stored.
As described earlier, the edited code is stored temporary in a JavaScript object representation. If the browser state was deleted at this stage, all changes would get lost.
Storing the code permanently happens in two stages.
First, when the programmer closes the Proceed IDE and returns to the BPMN editor, the ScriptingIde component updates the processEditorStore by turning the JavaScript object representation of the process, which contains
the modified code, back into an XML string, and committing it using its setXml setter.
Before it does that, it bundles the code in each JavaScript object representation of any script task with the library. This is done to create independently executable scripts within each script task which don’t require the process engine to load or combine any scripts to run each script task including all library functions. This is done by creating a simple module bundler inspired by an article by Adam Kelly. Being the simplest form of a module bundler, all it does is merging the library and script task code within a self-calling anonymous function:
(function(){
/************** LIBRARY BEGINS **************/
/************** LIBRARY ENDS **************/
/************** SCRIPT BEGINS **************/
/************** SCRIPT ENDS **************/
})();For example, if the following function is defined in the library:
function takePhoto() {
services.system().startCapability('PhotographAction',{
resolution: 720
});
}and the code of a given script task looks like this:
takePhoto();
next();then the code stored within the XML for this script task looks like this:
(function(){
/************** LIBRARY BEGINS **************/
function takePhoto() {
services.system().startCapability('PhotographAction',{
resolution: 720
});
}
/************** LIBRARY ENDS **************/
/************** SCRIPT BEGINS **************/
takePhoto();
next();
/************** SCRIPT ENDS **************/
})();This way, the script can be executed without polluting the global namespace, while having access to all library functions needed without requiring any additional code. To make sure all script tasks have the most current version of the library, every single script task in a given process is being re-bundled when closing the Proceed IDE. In return, while reading script tasks, the bundle must be separated again so that the editor only displays the script code, but not the library.
This is done whenever a script task is being opened. A simple regular expression separates the script code from the bundle:
The regular expression has been found to work very well. The greediness of the pattern matching even works when a programmer uses the same comments used as separators within their code. It only flaws when any of the comment separators are used within the library code, in which case it starts matching too early. To avoid this problem, any usage of the exact same comment string within the library is removed before bundling as follows:
library = library.replace(
/\/\*{14} (SCRIPT|LIBRARY) (BEGINS|ENDS) \*{14}\//g,
'');TODO: Delete IIFE around script tasks
While all updated code is now in the processEditorStore, this state does not persist. This is due to the fact that the processEditorStore is re-created from the permanent processStore whenever the BPMN editor is opened. This way, changes to the processEditorStore still don’t survive a page refresh, for example.
In order to permanently save all changes to the BPMN XML, including any changes to the scripts, the programmer must click the “save” button in the BPMN editor.
This calls the BPMN editor’s saveXml function, which commits all changes to the processStore.
Before that, it calls the saveIntermediateXml function which does a few preparations to the script tasks:
- it adds the necessary “scriptFormat” attribute to the script tasks
- it wraps all scripts in
CDataobjects - it includes all required Capabilities in the XML as described below
Apart from storing the capabilities in XML, the code to permanently store the script tasks in the processStore was already included in the Proceed Management Engine before the start of this thesis.
After the programmer decides which capabilities-parameter combinations are required to run their code and saves the BPMN diagram, the capability mapping must be stored as well. When editing the capability picker, those values are immediately updated in the processEditorStore within the elementCapabilityMapping. When saving the BPMN diagram, this is automatically stored in the processStore.
When all changes to a BPMN process are being saved to the permanent processStore, the mapping between script tasks and capabilities is stored within the BPMN XML document according to the requirements in the Proceed documentation. Below is an example of how the capability requirement for a “PhotographAction” with a required parameter “resolution” is stored:
<proceed:capability capabilityShortName="PhotographAction">
<proceed:cParameter>
<proceed:cValueOwnName>duration</proceed:cValueOwnName>
</proceed:cParameter>
</proceed:capability>When the Proceed IDE is reopened, it reads the capability mapping from the processStore. The capability XML is ignored and overwritten on save.
While the library is already being stored as a copy within each single script task code bundle, it needed also be decided how the library itself is stored. This is important, so the library is always available even if there is no process with no script tasks. (Option 4 was selected.)
See the other options...
Option 1: Script Tasks
One option is to not additionally store the library at all, since it is already included within each script task bundle and can easily be separated from it. It is a lean approach which requires no additional storage or markup overhead. The library can simply be parsed from any script task within a process. Since they are all re-bundled when the library is updated, there is no need to worry about discrepancies between the libraries of different script tasks.
However, a major downside from this approach is that the library is deleted with the removal of the last script task element of a process. Then, there is no copy of the library any more. When the programmer now adds a new script task, they will have to start the library from scratch as well, which most likely is not an intended outcome.
Option 2: Separating Library Script Task
In order to store the library code separately, it could be stored within the process XML. One way to do this without altering the markup structure is to simply use any unique script task called “Library” within a process as the library. The programmer may create a standalone script task element outside of the main process flow, call it “Library”, write functions and they will automatically be bundled with all other script tasks.
The library for the currently open process is cached in the libraryValue data element in the scriptingIde component for all other components to use.
Whenever the library element is updated, a function updateLibraryValue is called which updates the local cache:
updateLibraryValue() {
let libraryValue = '';
// find first element which is called "Library"
const hasNodeValue = R.any(R.propEq('nodeValue', 'Library'));
let libraryElement = R.filter(R.compose(hasNodeValue, R.prop('attributes') ))
(R.filter(R.prop('attributes'))(this.processElements));
libraryElement = libraryElement[Object.keys(libraryElement)[0]];
if (libraryElement) {
const scriptElement = R.find(R.propEq('nodeName', 'bpmn2:script'))
(libraryElement.childNodes);
libraryValue = scriptElement && scriptElement.firstChild ? scriptElement.firstChild.data : '';
}
// store in data
this.processLibrary = parseCodeFromBundle(libraryValue);
}It finds the first script task within the currently open process which is called "Library", finds the data stored in its child node and sets the local processLibrary cache to the new value.
Option 3: Process XML
Another option is to store the library within the BPMN XML to extend the XML structure. This has the benefit, that the full library is available separated when the BPMN XML is transferred to another Management System.
This, however, requires yet another new set of XML definitions and more markup that isn’t strictly needed for the execution of the process by any engine, and thus just adds overhead.
Option 4: Management System
The other option is to save the library additionally to the bundles in a separate place, for example within the database of the Management System. This approach was considered because it also doesn’t require changes to the markup and XML schema of the BPMN process, while always having access to the processes library and keeping it in a separate place to the bundles.
The downside of this approach, however, is that in case there is no script task currently in the process, the library is not available in any other Management System where it may be opened, which contradicts the idea of providing the developer with a library to save time.
Yet, this option was picked to provide a simple solution which doesn’t add overhead to the markup of the process BPMN XML. The library is saved within the processStore and committed to the database together with the process item. The library of the currently open process is also loaded into the processEditorStore, where it can be edited by the Monaco editor. The scriptingIde component loads the library as the first menu item for each process, and for all other processes apart from the one currently opened, they are read-only.
A key aspect of the user interface of the Proceed IDE is to give as much screen estate to the developer as possible. Otherwise, it will be very cluttered. Thus, the sidebar of the menu had to be removed.
There are three ways to display the Proceed IDE in a separate, full-screen environment.
See the possibilities to open a New Page or a New Window
The Proceed IDE can be opened as a new screen by navigating to a separate route specifically for the Proceed IDE. The sidebar menu can be hidden for this route, while a “back” button guides the programmer back to the BPMN editor.
This is a resource saving solution, since the state of the BPMN editor will be discarded. At the same time, this is not wanted since none of the changes of the BPMN editor are finally saved yet, and are needed to not lose changes to the BPMN document without saving them first, and have them available within the Proceed IDE.
Another option is to open the Proceed IDE in a new window. This has the additional benefit, that the programmer can open the IDE while still having access to the entire BPMN editor and all other resources within the Proceed Management System and can look them up even better than in the info tabs in the sidebar, especially in an environment with two or more monitors.
Electron allows opening new windows and pointing them to a route in the application:
openScriptDialog() {
const selectedElement = this.$store.getters['processEditorStore/selectedElements'][0];
if (selectedElement.type !== 'bpmn:ScriptTask') {
return;
}
const modalPath = process.env.NODE_ENV === 'development'
? 'http://localhost:9080/#/scripts/${selectedElement.businessObject.id}'
: 'file://${__dirname}/index.html#scripts/${selectedElement.businessObject.id}'
let win = new remote.BrowserWindow({ width: 800, height: 720, webPreferences: { webSecurity: false } });
win.maximize();
win.on('close', () => { win = null; });
win.loadURL(modalPath);
}The code above could be executed within the BPMN editor when a programmer selects a script task in the diagram and clicks the “Edit Script” button.
It finds out the currently selected element from the processEditorStore and creates a new Electron.remote.BrowserWindow instance, maximizes it and loads the route of the Proceed IDE.
It also passes the ID of the currently selected element as a parameter so the Proceed IDE can open it straight away.
However, this solution inherits the problem of opening the IDE in a new page, since it also must navigate to a separate route. Even worse, apart from synchronizing with a database, no state can be shared between the two windows making it even harder to synchronize edits between the BPMN editor and the Proceed IDE. Furthermore, any simultaneous changes to both the BPMN diagram and the Proceed IDE would overwrite each other when committed to permanent storage.
The last option is to open the Proceed IDE in an overlay from within the BPMN editor. When a programmer selects a script task and clicks its “Edit Script” button, the Proceed IDE component is loaded within a full-size overlay.
This way the route of the application stays the same and the entire state of the BPMN editor is kept and can be passed down to the Proceed IDE by implementing the Proceed IDE as a child component of the BPMN editor. This way, even attributes which are not committed to any VueX store can be accessed by the Proceed IDE. In return, any data can be passed up to the BPMN editor directly via Vue events.
The downside of this approach is that the programmer can not at the same time have the Proceed IDE and the BPMN editor open to edit both the BPMN diagram and the scripts simultaneously. This is only possible by constantly opening and closing the Proceed IDE.
However, switching between the BPMN editor and the Proceed IDE is very quick, because both are
loaded at the same time.
Opening the Proceed IDE is only a matter of setting the v-show flag of the Proceed IDE overlay to true, which is instant.
For these reasons, especially the architectural limitations of the Proceed Management System and the limitations to transfer state between browser windows, the overlay approach is implemented in the final solution of this thesis. To sum up this sections’s findings, the full-screen view of the Proceed IDE is implemented by opening an overlay within the BPMN editor in order to freely share data between the BPMN editor and the Proceed IDE.
Various other features are implemented within the Proceed IDE to support the programmer with writing JavaScript-based script tasks within BPMN processes.
While writing script tasks, programmers often write long function calls in order to start a capability or read and set a process variable. Most of the time, the programmer looks up the capability or the variable in question within the Devices or Variables tab. Thus, an inserting functionality is implemented within the Proceed IDE which pastes sample code at the current cursor position in the editor.
Whenever a programmer clicks on a variable or capability within those tabs, a function creates sample code for their usage and emits an event to the parent component including that code.
The following code shows an example of how this is done for capabilities.
The insertCapability function is called when clicking on a capability:
insertCapability(capability) {
this.$emit('insert', createCapabilityFunctionString(capability));
}The createCapabilityFunctionString function returns a capability call function with all the capability’s parameters.
When provided, the function sets the parameter values to the default value, or sets them to null otherwise for the programmer to customize the values themselves.
The parent component of both tabs, which is the ScriptEditor component, captures these events using the insert function:
insert(text) {
if (this.readonly) {
return;
}
this.$refs.editor.insert(text);
}It takes the text bubbled up by the “insert” event from the tab and calls another insert function on the MonacoEditor component which is stored as a reference called editor, passing the text as a parameter.
The insert function of the MonacoEditor component then finds out the current cursor position of the editor, and pastes the code from the parameter using the executeEdits function from the Monaco API as follows:
insert(text) {
const p = this.editor.getPosition();
this.editor.executeEdits('', [{
range: new monaco.Range(p.lineNumber, p.column, p.lineNumber, p.column),
text
}]);
},This is how pasting sample code into the Monaco editor from a tab in the sidebar is implemented, speeding up the use of capabilities and variables for the programmer.
Another way to help a programmer write code faster is to provide suggestions for Proceed’s functionalities while the developer is typing. As part of this thesis, two kinds of resources are made available as suggestions to the programmer: capabilities and variables.
A cached version of all available devices with their capabilities is stored within the ScriptEditor component. This list stores the capabilities nested under their devices, but in order to suggest all capabilities a flattened list of capabilities is created using a computed property:
capabilitiesInEnvironment() {
return [].concat(...this.devices.map(
device => device.capabilities || []
));
}The function takes the capabilities of each device and concats them together in a single array. The variables in turn are already available in a single array, thus a computed property simply takes them from the open processes processStore.
The function refreshSuggestions which is being called periodically by the ScriptEditor component then creates suggestion objects from both the capabilitiesInEnvironments and the variables array and stores them in a local array called suggestions.
A suggestion object for a variable can look as follows:
{
label: 'sampleProcessVariable',
insertText: 'services.getProcessVariable("sampleProcessVariable")',
detail: 'text',
kind: 'Variable'
}The insertText value is generated by the same functions which generate sample code for pasting capabilities or variables into the Proceed IDE.
The entire array of suggestions is passed down to the MonacoEditor component using the suggestions property. When the component mounts and the Monaco editor is being set up, these suggestions are added as a custom completionItemProvider with registerCompletionItemProvider as follows:
monaco.languages.registerCompletionItemProvider('javascript', {
provideCompletionItems: (model, position) => {
// must map to get rid of unwanted attributes generated somewhere,
// which implicate parsing by monaco editor and autocomplete not working correctly
const suggestions = self.suggestions.map(suggestion => ({
label: suggestion.label,
kind: monaco.languages.CompletionItemKind[suggestion.kind],
insertText: suggestion.insertText,
detail: suggestion.detail
}));
return { suggestions };
}
});When registering the completionItemProvider, the suggestions from the array are passed.
The kind of the suggestion, e.g. Variable, is transformed into a Monaco specific type.
Now, when the programmer starts typing sampleProcessVariable a suggestion overlay pops up next to the cursor and when the programmer hits enter, the sample code is pasted in to the editor at the current cursor:
A useful tool to help the programmer avoid mistakes are warnings and errors.
Monaco's built-in language service highlights JavaScript syntax errors.
However, there are also Proceed-specific warnings which can be shown to programmers, such as a lack of a required next() call or a capability call which cannot be executed on any available device.
TODO: replace with return for new BPMN Engine
Warnings are being displayed on the left margin of the code editor in form of a yellow, triangular warning icon or a red, circular error icon. When the programmer hovers their mouse above the icon, a warning or error message is shown:
Warnings are generated within the ScriptEditor component by the updateMessages method.
The method does two things:
- First, it checks whether the programmer uses a
next()call in their code. This call is required to tell the execution engine that the current script task is finished and the next task can be executed. If this call is lacking, which is checked by a simple regular expression, a warning message for the last line of the code is generated. - Secondly, it checks for each capability call, if there is at least one device in the network which could potentially execute it. If there is none, a warning message is added for the line of the capability call.
The entire array of messages is passed down as a property to the MonacoEditor component, which passes them to the Monaco editor as Decoration objects using the deltaDecorations API function.
The last tab in the right sidebar displays a list of available process variables. Since process variables are not made available by the processEditorStore, they are taken straight from the processStore.
TODO: change that => variables also to processEditorStore first.
Since the tab also allows the programmer to add, edit and remove process variables, these changes are directly committed back to the processStore. It is not required to save the process, the updated information is immediately committed.
When the programmer clicks on any given variable, its getter function is pasted into the Monaco editor using the pasting mechanism explained above.
The variables tab is only shown for script tasks of the currently open process, since the variable overview is not necessary for read-only tasks.
To understand the RAM usage of the Proceed IDE and how it differs from the state before the development of the IDE started, comparison tests were run against the last commit which only had an empty text field as an editor.
To test both how much more RAM the IDE itself needs, and how they both behave with a scaling amount of BPMN nodes, snapshots of the heap at various points were taken:
In an idle state, meaning without any processes or nodes present, the Proceed IDE takes up more than twice of the RAM the previous editor needed. This is a big change, but one needs to keep in mind that the Proceed IDE was not the only significant change made between the commit used for comparison and this project’s final code at time of writing, as other users were committing their work as well.
The figure shows two diagrams, one shows the absolute amount of RAM needed by the application depending on the amount of nodes in the BPMN diagram. The right diagram shows the amount of RAM needed for the given amount of nodes relative to the amount needed by the application in a state without nodes in the diagram.
The new Proceed IDE not only needs 57,2 MB more RAM without any script tasks in the BPMN diagram, but also needs 12,7 times more RAM per additional script task than the old editor. This is a big increase, most likely due to the fact that the ScriptingIde component always keeps an open copy of the entire XML diagram in object representation in its data set. However, this increase in memory usage is mitigated by the fact that it is still a linear increase in memory per each node.
TODO:
The memory hunger of the application can probably be mitigated by not operating on a full object representation of the entire BPMN XML file of all available processes but lazy-loading opened files into small objects which only contain the data needed for the Proceed IDE.
On every single keystroke of the programmer, analysis is performed on the entire code of the currently open script task. This way, warnings and suggestions are generated and used capabilities detected.
To understand the performance of this feature, it was benchmarked by capturing microsecond timestamps around the analyzeCode function within the ScriptEditor component.
Those were then logged in the console and noted.
Since the most analysis is done on detecting capabilities and matching them to existing devices, an automated function was written which pasted the following capability call code concatenated a certain amount of times into the editor, and then benchmarked the analysis of that code:
services.system().capability().startCapability('https://schema.org/PhotographAction', {
'https://schema.org/width': 1200,
'https://schema.org/height': 800
});The test was run on a 2019 Macbook Pro 13” with a 2,7 Ghz Intel Core i7 and 16 GB RAM. The following figure shows the execution time needed to analyze code with a certain amount of concatenated capability calls. One capability call equals to 5 lines of code.
As depicted in the following figure, the execution time increases exponentially with the amount of capability calls within the code.
This is a big performance issue, even though it is questionable how many developers will write several thousands of capability call functions within one single script task. Still, since the code analysis function is called every time the developer writes a single symbol in the code editor, the UI may noticeably freeze from around 1000 capability calls, or much less on weaker devices.
To prevent that, the analysis function was debounced when called because of a value change, which essentially fires on every single keystroke of the programmer. Debouncing a function call is a means of ensuring a function is only called once within a certain amount of time. This way, if for example a function is debounced to only be called once within five seconds, even if it is called 3 times it will only execute once. Debouncing the code analysis function ensures that while a developer is typing fast, the code analysis will only run every 300 milliseconds. The debounce function was taken from the Underscore.js library and slightly modified to fit the linting standards of the project.
To fully solve the problem of the blocking UI, a separate worker would have to be created to run the code analysis in a separate thread, which would ideally run on a separate processor core and thus not interfere with the rendering of the interface.
More sophisticated IDEs offer more complex linting than just syntax error checking. Libraries like ESLint[33] offer very customizable sets of rules that programmers have to follow - otherwise warnings or errors will be thrown. Since ESLint is a JavaScript library itself, it can be integrated into the Proceed IDE. It makes for a logical next step.
In order to permanently save code within the Proceed IDE, at this moment, the programmer needs to close the IDE which bundles the code and updates the BPMN XML, and then save the BPMN diagram. These are many steps to save code, most IDEs offer a simple save function via the keyboard shortcut “Ctrl+S”. Furthermore, the Proceed IDE cannot save single files independently, but always saves all changes at once.
A function to independently save single scripts from within the IDE will be a useful, time- saving feature, which could be further improved by auto-saving the code regularly to prevent data loss.
While not a classic task for a script editor, file bundling does belong to the tasks needed by
a programmer within their development environment. Since the programmer cannot structure their code in multiple files within Proceed at the time of writing this thesis, a sophisticated
bundler which builds a dependency tree and makes dependencies available via require statements was not needed.
Should a more complex file structure get implemented, a more sophisticated bundler like webpack should be used as well.
Another important aspect for assuring a programmer’s code quality are unit tests. Modern IDEs support programmers with running unit tests and pointing out where they fail. Since Proceed is not just an IDE but an entire environment and runtime, it would make most sense to not just provide an interface for running unit tests, but also an entire framework to mock state, capabilities and variables. This would ensure that script tasks fulfill all requirements and that changes within library code don’t break any script tasks.