[FEEDBACK] List level handling, Stethoscope, and error messages

[sna-scourtney]

This is not a bug report, because I am absolutely aware it's user error on my part. Rather, I would like to offer some constructive feedback about learning Sverchok as a newcomer, and some suggestions about improvements that I hope could be made with some additional defensive code in very generic parts of the system -- that is, one change that could affect usability of many nodes. Let me preface this by saying I have fallen in love with Sverchok, and I'm looking forward to getting my first contribution shared onto the gallery (I am making a procedural multi-room generator for office areas and similar spaces).

The general issue is list level handling, though my illustration is the List Slice node, which I've been struggling for multiple hours because of edge cases in the data. In these comments, I am going to throw random numbers into example lists, with the understanding that the numbers themselves have no significance and are arbitrary.

Generally speaking, I've found it difficult -- even with a programming background -- to be sure what list structures are emitted by certain nodes, or expected by the next node, and the error messages and Stethoscope don't always present the data in a way that's easy for new users to understand. For example, if I use the A Number node to emit a simple integer, the value is shown by Stethoscope as [[5]] -- and I can't be sure whether that's a two-level list being created for an integer constant, or if that's an artifact of Stethoscope itself being able to handle arbitrary inputs. In graphical mode, the Stethoscope shows that A Number is a "list of 1 items {int: 1}". When I look at the code for the Number node, I can see that it is emitting a simple constant. But while the code is clean and very readable to a programmer like myself, there are enough Sverchok core functions and OO constructs that this code would not be self-evident to a non-programmer.

My first suggestion, then, is that the Stethoscope node's output should show what it is really receiving as input -- if it's getting a constant rather than a list, or if the innermost elements of a list or list hierarchy are constants, this should be reflected. Is this a side-effect of an input stage that wraps constants with a single level list, to allow more uniform handling of all cases later in the node?

Second, I think one small improvement to error handling would make an enormous difference in user experience. As far as I can tell, the error messages resulting from unacceptable input values do not show which input is at fault. Sometimes it is evident from context -- for example, on the List Slice node an error saying the input cannot be subscripted is obviously referring to the one port expects a list. But for a "subscript out of range" error, there are two inputs that could be at fault (or perhaps both of them!), and it would really help if nodes displayed the port name(s) that caused the error. If more than one input is wrong, perhaps there could be one line of diagnostics shown for each one?

Finally, I'd like to suggest that the List ____ nodes and the View Draw diagnostic node get some "defensive code" to gracefully manage empty inputs. For instance, the List Levels node generates an error if it is asked to wrap an empty input, whereas this is conceptually meaningful. I have a List Slice node whose output is a list of floats [1.25, 2.75, 4.25] for example. That list may happen to contain one element such as [2.75] or it may legitimately be empty if the Slice was given equal values for Start and Stop. The List Levels node that follows the Slice node is asked to wrap level 0 and does so, but errors out for the empty list instead of emitting []. Many downstream nodes still make conceptual sense with zero-length input lists -- for example, List Join [1, 2, 3, 4] and [] should return [1, 2, 3, 4]. The View Draw node should accept empty inputs for vertices, edges, or polygons and should respond by disabling its output as needed (disabling everything if its vertex input is empty, of course).

As an aside, the Slice node uses the Python subscript range notation e.g., "array[start:end]" which is zero-based for the start and one-based for the stop, a fact known to Python programmers but not to non-programmers. It would be useful to mention this in the documentation (I don't believe I have access to add this myself). Again, this is clear to me from examining the source code, but it would be intimidating to a non-programmer. :)

I dislike making work for other people, so if there is any of this that you feel is worth considering, and I can be of help in implementing or testing, please let me know what you'd like me to do. :)

Sverchok is a terrific tool, and my suggestions are offered in hopes of improving a great thing, not in a negative way. Thanks for making this available to the Blender community.

[zeffii]

Nesting

it's not tricky, there's sanity behind what seems like madness.

Stethoscope

Stethoscope gives a true reflection of the data passed into it.

• [[5]] is correct for a single digit.
• two digits in the same list would be [[5, 5]]
• 4 digits spread over 2 lists would be [[5, 5], [5, 5]]

Debug Print

There's another node that you can use to see the data, text->debug print . You'll see their output is generally the same, but Stethoscope will filter out the middle of larger data sets using ellipses. The code for Debug Print node is relatively transparent.

There's a doc worth reading: https://github.com/nortikin/sverchok/blob/master/docs/tutorials/nesting.rst

[sna-scourtney]

Nesting

it's not tricky, there's sanity behind what seems like madness.

Stethoscope

Stethoscope gives a true reflection of the data passed into it.

• [[5]] is correct for a single digit.
• two digits in the same list would be [[5, 5]]
• 4 digits spread over 2 lists would be [[5. 5], [5, 5]]

I agree the contents are correct; my assertion is that the output format is counter-intuitive. Your examples appear to show two-level lists for all three, but are they actually? I would expect the first one to be '5', the second '[5, 5]', and the third exactly as you show it. Or am I still misreading this? The code of the Number node certainly appears to return a simple scalar constant, but is this somehow being inserted into a list further up the call chain?

Debug Print

There's another node that you can use to see the data, text->debug print . You'll see their output is generally the same, but Stethoscope will filter out the middle of larger data sets using ellipses.

There's a doc worth reading: https://github.com/nortikin/sverchok/blob/master/docs/tutorials/nesting.rst

I'll take a look at the Debug Print node -- thanks for that, and for the doc reference for the nesting. I had read that one as I first started, but perhaps I need to go back and refresh my knowledge.

[zeffii]

you have a github account, feel free to add suggestions as a PR to the list-slice node. I agree btw https://github.com/nortikin/sverchok/blob/master/docs/nodes/list_struct/slice.rst is not very inviting to a non programmer. this is my fault.

[zeffii]

but are they actually?

yes, It represents a single list with a single number, but there's an extra set of brackets to encapsulate it.

[zeffii]

think of it as a minor peace of boilerplate for our data-structures. Like a schema. but really really basic.

[sna-scourtney]

but are they actually?

yes, It represents a single list with a single number, but there's an extra set of brackets to encapsulate it.

In that case, then the Stethoscope is showing actual levels, and it was just not obvious that the Number node is emitting a single-element list. I reviewed the Nesting documentation you suggested. That was a very good suggestion, because it is more meaningful now than it was when I was first starting. :) The key concept seems to be that everything is a list, even in contexts where the naming of sockets would at first suggest otherwise. I'll keep that in mind.

One of the many things I like about Sverchok is its generality of data handling -- I just finished creating a group node whose function was designed around single-value input parameters, and yet I have easily made it work with list inputs for those same parameters. That generality is powerful.

[zeffii]

Nesting is one of those hurdles for new Sverchok users. The extra levels of nesting is exactly how generality is achieved (even if it seems excessive). Other approaches would be possible, but we settled on a simple base structure early. There are complicated backend list-consistency functions which get things right most of the time - for sane input.

The key concept seems to be that everything is a list, even in contexts where the naming of sockets would at first suggest otherwise.

yeah, it's not really sane to have super elaborate names for those sockets. Eventually, after the hurdles mentioned above, you want to see exactly the minimal socket name.

[sna-scourtney]

you have a github account, feel free to add suggestions as a PR to the list-slice node. I agree btw https://github.com/nortikin/sverchok/blob/master/docs/nodes/list_struct/slice.rst is not very inviting to a non programmer. this is my fault.

I'm taking your advice on this. I'll submit a PR for the documentation page first, which I think is a good place to start contributing. I have cloned the repo and also created an empty repo of my own for sandboxing. I should also mention that I'm reading the contributor guides before I begin, but if I accidentally break project conventions please let me know so I can improve.

BTW, I agree fully with your most recent comments about nesting and naming. As with many tools, there is "the Sverchok way of doing things" that new users must learn.

[zeffii]

I may have written that in 2015 and never looked at it again. (it seems to have lost some formatting in the meantime) It would be fitting to add a small python specific slice explanation with a link to the python docs, and how the raw python would compare with the Slice node.

[zeffii]

but if I accidentally break project conventions

if it's code, there will be many more eyes reading it :)

[Durman]

There are a lot of problems with how nodes work now. Handling empty lists and meaningful error messages are one of them. Number of Sverchok nodes is close to 700 now. To fix them all will take huge amount of time. It will probably even never happen.
The right strategy here is to move to the goal gradually. There is an issue #3955 about how to improve work of the nodes and there are bunch of nodes which have experimental implementation of some features described in the issue (Matrix apply node one of them). When ideal node will be found Sverchok should be able to handle new and old approaches simultaneously so old nodes could be slowly rewritten (probably not all of them but most important ones).

If you wish to take part in improving Sverchok in this direction you can choes one of the existent nodes and fix it in a way you think it should work. But keep in mind that fixes should not break existing layouts (after this PR it will be simpler #4253). There is for example the issue about exactly the problem of handling empty lists - #4187. I'm going to add contribution page later to put useful links for new commers to blender API and some others.

[enzyme69]

SVerchok and SV nodes "levels" have always been pretty strange and weird, just work out the LIST SPLIT and LIST JOIN and watch the data using stethoscope. Don't try to do advanced things, just plug simple random numbers and see how vectorized data being calculated.

Think of it like Attribute Math of Geometry Nodes, just a little bit more advanced: