Automatization of node creation

[Durman]

I have continued experiments with optimization API of node creation and reach much further last attempt.
Next code is working:

from itertools import cycle

import bpy
from sverchok.data_structure import updateNode

from sverchok.node_tree import SverchCustomTreeNode
from sverchok.utils.handling_nodes import NodeProperties, SocketProperties, SockTypes, WrapNode, initialize_node

node = WrapNode()

node.props.x = NodeProperties(bpy.props.FloatProperty(update=updateNode))
node.props.y = NodeProperties(bpy.props.FloatProperty(update=updateNode))

node.inputs.verts = SocketProperties(name='Vertices', socket_type=SockTypes.VERTICES, mandatory=True)
node.inputs.faces = SocketProperties(name='Faces', socket_type=SockTypes.STRINGS)
node.inputs.x_axis = SocketProperties(name='X axis', socket_type=SockTypes.STRINGS, prop=node.props.x)
node.inputs.y_axis = SocketProperties(name='Y axis', socket_type=SockTypes.STRINGS, prop=node.props.y)

node.outputs.verts = SocketProperties(name='Vertices', socket_type=SockTypes.VERTICES)
node.outputs.faces = SocketProperties(name='Faces', socket_type=SockTypes.STRINGS)


@initialize_node(node)
class SvTestNode(bpy.types.Node, SverchCustomTreeNode):
    bl_idname = 'SvTestNode'
    bl_label = 'Test node'

    def process(self):
        node.outputs.faces = node.inputs.faces
        node.outputs.verts = [(v[0] + x, v[1] + y, v[2]) for v, x, y in zip(node.inputs.verts, cycle(node.inputs.x_axis), cycle(node.inputs.y_axis))]

It should looks magically.

Most advantage of current approach is it impacts on nothing. It does not change/add any node base classes. But new nodes can gain from it.

Also I have tried to use autocomplete as much as possible. At present most names will be hinted by IDE. Access to sockets can be get via dot notation like that node.inputs.verts

It is possible to set which sockets should be vectorized and all other works will be done automatically. The process function should be coded as if Sverchok does not has vectorization.

In the end I expect to get less bugs and more new nodes coded, in the same time interval.

[zeffii]

i am in favour of an abstraction for the node boilerplate (this is what drove me to write ScriptNodes also)

this implementation can be abstracted a lot more i think, even though you mention the autocomplete...it's still pretty intense/dense.

[portnov]

Well, what does it abstract? This doesn't seem to be much shorter than our usual nodes... Maybe it is doing something behind the scene? :)

In my experience, most of boilerplate comes from data nesting / vectorization support... something like

out = []
for a, b, c in zip_long_repeat(...):
    new = []
    for a, b, c in zip_long_repeat(...):
        new.append(...)
    out.append(new)

The problems I see here are

1. it's too easy to write something like parameter = parameters[0] instead of additional for-loop, thus breaking vectorization;
2. it's not everything clear how many exactly there should nested for-loops be;
3. it's too easy to confuse order of out.append, new.append, or confuse them with out.extend, and so make incorrect nesting in the outputs;
4. The supported data level is hard-coded by number of nested for-loops you write; if for some reason you want the node to work with list of lists of lists of vertices instead of lists of lists of vertices, you have to modify the code and add additional for-loop; moreover, in such a case you have to think, whether do you want to add additional nesting level to outputs, correspondingly, because in most cases subsequent nodes will not be able to handle it...
5. you have to invent variable names for things like "list of parameter values", to be used only in one following line of code with for ... in zip_long_repeat(...).

In general, it seems to me that requirements for "good data matching" are relatively simple:

1. most of our nodes that produce numbers, such as "number range" output data with nesting level 2 (lists of lists of numbers); So the node should be ready to receive at least nesting level 2 in "numeric" sockets, and "list of lists of objects" in other sockets, to match one number with each "object" it relates to;
2. it should be ready to receive lower nesting level than it actually works with, automatically adding as many [] as necessary;
3. nesting level of outputs should correspond to nesting level of inputs; for example, if the node takes "list of lists of curves", and generates a list of vertices from each curve, then it should output list of lists of lists of vertices.
4. it would be good if the node could automatically handle data with arbitrary nesting level, automatically doing as many nested iterations as required.

It would be good to have something like

# This says that the node wants to handle one list of vertices together with one value of Parameter
matching = Matching(
                          Verts = Socket(level = 2), # Single vertex has level 1; list of vertices has level 2; any deeper level requires iteration
                          Parameter = Socket(level = 0) # Numeric parameter has level 0, any deeper level requires iteration
                     ...)
matching.mode = Matching.REPEAT_LAST
# This would:
# 1) match data and nesting levels in inputs, for example if the node receives data with different nesting levels in different inputs, this will add as many [] as required to match each list of vertices with one value of Parameter
# 2)  with data.next() it would iterate through the resulting data tree, doing as many nested "for" loops as needed;
# 3) with data.output[].add() it would manage output data lists so that in the end the data in output sockets will have correct nesting levels - in general, the same nesting level as in the inputs
with data_iterator(self.inputs, self.outputs, matching) as data:
    while data.next():
        new_verts, new_faces = do_something(data.input['Verts'], data.input['Parameter'])
        data.output['Verts'].add(new_verts)
        data.output['Faces'].add(new_faces)

[Durman]

this implementation can be abstracted a lot more i think, even though you mention the autocomplete...it's still pretty intense/dense.

@zeffii Yes. I see the next step in deleting SvTestNode class at all. The class can be created and registered automatically. But it will involve more coding and I decided to stop on current stage to see if necessity of it will be proved on practice.

Maybe it is doing something behind the scene? :)

@portnov It's doing. You can have a look at dissolving node code. I automated standard vectorization. So process method is calling separately for each loop. I think other vectorization systems can be considered but it would be good to get evidence that approach is good and worth further developing.

[nortikin]

that what i wanted to have but my knowledge is not enough to do as it should be written initially.
"Experience - the son of hard mistakes" ©️

[Durman]

I've added code for automatization of node creation along with Dissolve node PR. If someone interested in using it - welcome. The node itself can be used as documentation of using automatization.

[Durman]

Experiment continue.

I'm thinking about avoiding of setting update property function manually. This function is going to be added automatically.

Also sockets got extra show parameter with function which should return bool value. The function will have access to node properties so each socket can have its own condition of when it should be shown.

[Durman]

Comparison of code readability of index to mask node without and with automatization approach.

[Durman]

Also I have added auto registration of node classes. No need in writing register and unregister functions for each new node.

[nortikin]

Also I have added auto registration of node classes. No need in writing register and unregister functions for each new node.

how to update new node's text on fly than? i was used if __name__ == '___main__': register()

[Durman]

What do you mean on fly? If I'm adding new code to a node I just press reload Blender (I had binded the command to F8 button) and changes appear.

[nortikin]

What do you mean on fly? If I'm adding new code to a node I just press reload Blender (I had binded the command to F8 button) and changes appear.

there is run button in text editor with register flag, allows to run node without reload blender. it is faster.

[Durman]

You can try rerun this file. It is where classes are actually registering.

sverchok/utils/handling_nodes.py

Line 407 in 3e31747

register, unregister = bpy.utils.register_classes_factory(node_classes)

[Durman]

@nortikin I have tested if __name__ == '___main__': register() works as expected in file where node class is located.