Update system mem #2393

Durman · 2019-03-28T16:14:17Z

This is an attempt to remake update system. Main goal is to make such system that would calculate node only when this is really need. I think this can become really important update of Sverchok that will make nodeing a little bit easier.
There are 3 update events:

event	old US	new US
update node	updating only changed nodes	updating only changed nodes
update tree	updating every node in tree	updating only changed nodes
animation update	updating every node in tree	update only animation nodes and next after them

In new update system there are 3 conditions that determine should node be recalculated or not:

All sockets (inputs and outputs) should have the same links as previous update.
All data passing from input sockets should have the same data as previous update event.
All parameters of node should be the same as previous update.
In most cases this approach should work faster and even much more faster.

This branch is ready to test. In this moment monads is not supported yet the same as nodes which takes data from blender database excluding frame info node.

Testing instruction:

First of all old update system should be disable in Svechok preferences.

After that you can start testing. In node tree changed nodes will have red color. If you will notice some nodes that should not be updated but nevertheless has red color please make bug report in this topic.

It is possible to get additional information about process of update system:

For this you should enabled debug level of logging in Svechok settings and toggle log to text buffer.
Now in "Sverchok log" text file updating information is available.

For better perfomance of animation I would recommend disable logging.

#2380

Preflight checklist

Put an x letter in each brackets when you're done this item:

Code changes complete.
Code documentation complete.
Documentation for users complete (or not required, if user never sees these changes).
Manual testing done.
Unit-tests implemented.
Ready for merge.

* added gcode node * description corrected * fixed continuous extrusion * added credits * added feed movements * new formula * close all shapes * list matches and path preview * text format * removed one output socket * Added Evaluate Image Node and fixed limit in Image Node * removed comments * CLIP and EXTEND tile mode in Evaluate Image Node * removed unnecessary code

For some screen resolutions (e.g. mac book pro with retina display / double pixel resolution) the index viewer node's font size was too small to read and needed a scale factor of 2x. With this new setting the user can define the desired font scale in the SV preference settings. The default font scale is 1.0.

- made scale and xy multiplier pref settings generic to be used by multiple nodes - add scale/multiplier adjustments to the texture viewer and easing node and update stethoscope node to use the same scale/multiplier

* line mk3 with AB and OD modes * updated tests to mk3 * lime mk3 docs

* Added Alpha and BW grayscale * fixed grayscale

Profiling the edge generation indicated that it is a slow operation and since this edge list is generated over and over in various nodes it makes sense to cache it. Profiling of various nodes (generating edges) indicated a speedup of almost two orders of magnitude in generating the edges compared to the list comprehension counterpart (e.g. edges = [[i, i+1] for i in range(N)]. With the proposed optimization, the list of edges are stored in an edge cache (e.g. [[0,1], [1,2]... [n-1,n]] .. and the cache is extended as longer edge lists are requested/generated. Any call to get_edge_list will return a subset of this edge cache thus not having to re-generate the same list over and over. Various nodes like the line, spiral, torus knot, ellipse etc, which that generate lines with linear list of edges can benefit substantially from this speedup. To get the linear edge list use: edges = get_edge_list(n) returning: [[0, 1], [1, 2], ... , [n-1, n]] To get the loop edge list use: edges = get_edge_loop(n) returning: [[0, 1], [1, 2], ... , [n-2, n-1], [n-1, 0]]

Optimize the linear edge list generation using caching

…al angle

New PolygonSort node

…ing the following: Sum Sum Of Squares Product Average Geometric Mean Harmonic Mean Standard Deviation Root Mean Square Skewness Kurtosis Minimum Maximum Median Percentile Histogram

Add new List Statistics node

Port the same preference settings for xy/scale into the TVL node to fix the location and the size of the rendering.

Add scale/multiplier to TextureViewerLite node to fix rendering

The combinatoric operations implemented so far, based on itertools module are: PRODUCT PERMUTATION COMBINATION

Add new Combinatorics node

- add 3 nodes: Quaternion In, Quaternion Out and Quaternion Math Quaternion In: Provides conversion from various quaternion components to quaternions (selectable via the mode option): - components wxyz - angle + axis - euler angles - matrix Quaternion Out: Provides conversion from quaterions to various quaternion components (selectable via the mode option): - components wxyz - angle + axis - euler angles - matrix Quaternion Math: Provides various quaternion arithmetic operations: ADD SUB MULTIPLY DIVIDE ROTATE DOT DISTANCE NEGATE CONJUGATE INVERT NORMALIZE SCALE MAGNITUDE Note: some operations take multiple quaternion inputs (input list grows as new sockets are connected), two quaternion inputs, single quaternion inputs, or quaternion + scalar input.

Add Quaternion nodes

…tures The node will mix various other types (not just int/float) hence the name change. Add various types to the MixInput node + new features & update UI - add new types to mix: Vectors, Colors, Quaternions and Matrices. - add SWAP feature to let the input A & B be swapped - add MIRROR feature to mirror the interpolation factor - add custom icon to MixInputs node - relocate old MixNumbers node to old nodes - update documentation

Formula node mk3

…th other nodes

Extend the features of the first cylinder node. This includes: - show/hide top/bottom caps separately - add twist to the parallel cuts - add overall phase to the parallel cuts - add parallel and meridian profiles to modulate the parallel loops and meridian lines - add overall scale - add centering option

Add new Cylinder node MK2

#2386

deleting unused operator

… on N panel

vicdoval · 2019-05-07T08:26:57Z

This looks awesome!
Just some questions:

When animating shouldn't be updated all keyframed nodes also?
In the Pulga Physics node I have created "Accumulative" mode. When activated the node stores the output and uses it as the input in the next update. To perform many updates I play the animation to make the shape evolve. This behaviour wont be possible with this new update system except if there where some way to create exceptions (if something then update always)
You think that could be included?
Because of the size of this update wouldn't you prefer to aim to the 2.8 branch?

Durman · 2019-05-07T16:21:36Z

Only frame info nodes are taken in account during animation at present.
Well, you can create additional property with value of random number for example. When your node is updating first time change this value if the node should be updated next time. Also you should keep in mind that there is at least three types of events of update system. This way will work only when the node tree was changed (click on any socket of any node in the tree).
At present I stucked on how to detect fastly was input object from scene updated. Probably this branch will remain experimental. I created this pull request only to share my thoughts about improving the update system. If it is closer to workable version I will consider about 2.8

zeffii and others added 30 commits November 3, 2018 14:03

adresses bug in upstream list modification

b72083b

add some error prevention

c92ebe0

Fix render scale and location of easing and texture viewer nodes

c5551c9

- made scale and xy multiplier pref settings generic to be used by multiple nodes - add scale/multiplier adjustments to the texture viewer and easing node and update stethoscope node to use the same scale/multiplier

line mk3 with AB and OD modes (#2294)

4a85936

* line mk3 with AB and OD modes * updated tests to mk3 * lime mk3 docs

small prophylactic for no output

65f187f

Added "BW" Grayscale and "A" Alpha channel support (#2314)

ab50e95

* Added Alpha and BW grayscale * fixed grayscale

Update evaluate_image.py

065af54

Merge pull request #2321 from DolphinDream/edgeGenerationSpeedUp

f3379f2

Optimize the linear edge list generation using caching

smooth about poligons

503e629

Update smooth.rst

f5424a4

Add new PolygonSort node to sort polygons by distance, area, and norm…

9992a5b

…al angle

Merge pull request #2324 from DolphinDream/polygonSortNode

1a4622e

New PolygonSort node

fixes lobsided edges (#2330)

987fc40

added Tension Node

98b1632

Update tree_generator.py

88c92e9

Add node to compute various statistical quantities, currently support…

421d246

…ing the following: Sum Sum Of Squares Product Average Geometric Mean Harmonic Mean Standard Deviation Root Mean Square Skewness Kurtosis Minimum Maximum Median Percentile Histogram

Merge pull request #2334 from DolphinDream/statisticsNode

2f660fa

Add new List Statistics node

Add scale/multiplier to TextureViewerLite node to fix rendering

5f6c38d

Port the same preference settings for xy/scale into the TVL node to fix the location and the size of the rendering.

Merge pull request #2343 from DolphinDream/textureViewerLiteRenderFix

ca22b47

Add scale/multiplier to TextureViewerLite node to fix rendering

Add new combinatorics node (WIP)

ad968de

The combinatoric operations implemented so far, based on itertools module are: PRODUCT PERMUTATION COMBINATION

Add documentation + optimize and cleanup code

321544a

Merge pull request #2346 from DolphinDream/combinatoricsNode

72896c4

Add new Combinatorics node

Fix comment

e6251c9

Merge pull request #2341 from DolphinDream/quaternionNodes

9df2933

Add Quaternion nodes

Renamed Deformation, cleaned and docs

c8c8e0f

portnov and others added 20 commits March 8, 2019 12:44

Support for easy migration from formula2 to formula3.

763ad51

Merge pull request #2379 from nortikin/formula3

6a094ce

Formula node mk3

Minor comments & code updates to SuperEllipsoid to keep consistent wi…

6ebff79

…th other nodes

Various optimizations, cleanup + documentation

50f7d01

Add angle units option

d542b3c

Merge pull request #2377 from DolphinDream/cylinderNodeMK2

c0af4e8

Add new Cylinder node MK2

deleting unused operator

38803f9

#2386

Merge pull request #2387 from nortikin/del_SverchokUpdateObjectIn

899a0a8

deleting unused operator

testing work wia PyCharm

14af970

New update system module

c667d6e

new setting for switching between old and new update system

a0549b8

improving of setting color to a node

8bbfb6f

connection new update system to update events

2578320

connection new update system to buttons update all and update current…

b1bd5a6

… on N panel

new approach to switcher "show laout" for new update system

3401e9e

adding logic to process layout switcher on N panel

a5e3036

external properties - frame_info_mk2

19ef0a6

support reroute nodes

09ff6f1

catching errors nodes

31293be

portnov added this to the After 2.8 milestone Jun 3, 2019

portnov mentioned this pull request Nov 11, 2019

Can we use Numba?... #2646

Closed

portnov added the Performance 🚀 label Nov 14, 2019

portnov force-pushed the master branch from 61deb2f to 2dbfffc Compare January 12, 2020 06:33

Durman closed this Feb 4, 2020

Durman mentioned this pull request May 7, 2020

New update core #3179

Closed

29 tasks

Durman deleted the update_system_mem branch October 5, 2020 12:37

Durman mentioned this pull request Jun 12, 2021

New update system #4164

Merged

16 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update system mem #2393

Update system mem #2393

Durman commented Mar 28, 2019

vicdoval commented May 7, 2019

Durman commented May 7, 2019

Update system mem #2393

Update system mem #2393

Conversation

Durman commented Mar 28, 2019

Testing instruction:

Preflight checklist

vicdoval commented May 7, 2019

Durman commented May 7, 2019