Nodes can block the UI #80
Replies: 3 comments 1 reply
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
As always, thank you for the feedback. Also, before I address this feature request, let me just add that since there's been a lot of replies/interactions here on the discussion tab in the past few days, I'll likely take a bit more time to reply to all of them. Also, because of my work and commitments outside open-source, my time is often limited as well. Regardless of all this, I'm glad for this many replies/interactions and I'd like you to know that I'll reply to everything eventually. So please, keep sharing your doubts, feature ideas/requests, even your concerns and constructive criticism as much as you want. This helps improve Nodezator and is also very interesting and instructive. Regarding your feature request, it is a very understandable and reasonable one, after all, it is just the norm when it comes to graphical user interfaces. It is not only a request that I accept, but something that has already been in my sights for a long time. I'm glad you found a workaround for now, as hacky as it may be. I regret not being able to provide this very basic feature to the users from the beginning, despite having a good reason for this, because I know that it is something that harms usability a lot as you aptly described. The need for multi-paradigm concurrent/parallel graph executionI'll now present the reason I have not been able to offer any sort of asynchronous graph execution for Nodezator. As we know, Nodezator is a generalist/multi-purpose Python node editor and each node represents an atomic/independent operation. It is as if each node is an app itself. The problem is that concurrency/parallelism comes in many forms, even more so in Python. Single-threaded concurrent code using coroutines (asynchronous programming) and threading, both paradigms for I/O bound tasks. Multiprocessing for CPU-bound tasks. There's also GPU-accelerated tasks that are also a form of parallelism. The mere mention of any of these subjects in isolation is enough to make the skin of any sane programmer to crawl. Now imagine that I have to deal with all of these subjects at the same time (no pun intended). I've been researching these topics for quite some time now, but I'm faced with the need to specialize in all of these topics before I can even conceive a solution for how to harmonize such paradigms, let alone start implementing anything. Additionally, most if not all available technical materials regarding concurrency/parallelism in Python approach problems by choosing one of these paradigms from the very beginning. Although I still have much to study/research I've never found any material dealing with problems from all available paradigms at once. It is always "if it is I/O-bound, use threading or asyncio, if it is CPU-bound, use multiprocessing", or similar things. Despite my frustration, I can't even blame such materials or their authors, because the approach is a very reasonable one for most Python problems. The problem faced by Nodezator is different though: in Nodezator, each node is an atomic, independent operation whose synchronous/asynchronous nature is (AFAIK) impossible to predict. How am I supposed to know if a node's execution is I/O or CPU-intensive? The answer is that I not only need to design an execution algorithm capable of dealing with operations with different concurrency/parallelism paradigms, but I'll probably also need to design a way for users to communicate it in their node scripts, so Nodezator know which kind of paradigm to use when dealing with each operation. I must also account for the possibility that the user doesn't want Nodezator to handle the concurrency/parallelism at all, but instead the user may want to deal with any concurrency/parallelism inside the node itself. That is, the node could trigger several concurrent/parallel operations while executing and only returning after make sure such operations finished. This is actually possible right now, depending solely on the proficiency of the user with the desired paradigm. That is, despite the fact that Nodezator's graph execution is synchronous, with every operation blocking, the users also have the option to access and control the application loop entirely from within their nodes when they are executed, if desired. For instance, the Thanks to pygame/pygame-ce very bare-bones but powerful API, even the largest example in those chapters only have a bit over 500 lines (and that's cause we are counting the comments as well, which are a lot). This is a very little line count, considering the example adds mouse and keyboard control and window resizing detection. It all depends on the user's Python and pygame proficiency. The Of course, adding concurrency/parallelism to a node like those we just presented should be possible with not many additional lines, but it would still require Python concurrency/parallelism expertise. Although such nodes don't need concurrency/parallelism, if there were tasks to be executed in such manner, it should be possible to implement all the subsystem needed while still preventing the app to hang by keeping the loop running while the concurrent/parallel tasks execute, possibly showing the progress of the tasks on the screen. You may not even need many nodes like this, only a few that could benefit from concurrent/parallel execution, and could thus keep most of the nodes in your node pack simple and free of concurrent/parallel execution. I never delved into that myself actually, that is, pygame+concurrency/parallelism. This is so because I never worked with a problem that required such advanced features and because I simply didn't have the time to try this yet. I actually only began to learn and use Python seriously in 2017, before that I was only able to use basic features and make small CLI apps. Since then, I've mostly ever worked in relatively large-scoped apps like games and related editors (level + animation editors), then Nodezator. My Python experience is not better than others', it is just that my approach ended up a bit different. Instead of learning a bit about a lot of topics and trying small projects, I started by learning only the basics and venturing to make larger projects. Naturally, as a result I had to learn several different topics in order to provide the needed services/features for my apps. It just have yet to delve deeper into the realm of concurrency/parallelism in Python. As I mentioned, I already started studying/researching it. I'm able to make small solutions. Nodezator, with its scope/purpose however, is an entirely different beast. It has to be able to use multiple paradigms. Because Nodezator was designed so that everything can be a node, we must also pay the extra price of also designing a proper multi-paradigm concurrency/parallelism execution model that can deal with all the required complexity. This isn't even everything. There's yet more complexity to be dealt with. First, the introduction of subgraphs/group nodes will also need to be taken into account in such multi-paradigm graph execution system. Second, since Nodezator was designed to be able to export its graphs as pure Python code, we may also need redesign/update the feature, though for know I'm not sure this will be needed. Tying everything together, Nodezator purposefully still sticks with a synchronous paradigm for graph execution because the the design of a concurrent/parallel solution needs a lot of study/research and careful consideration. It also must take into account features that are not even implemented yet, like subgraphs/group nodes, and which will undoubtedly shape the solution. Because of all this, it will still take sometime to concurrent/parallel graph execution to land in Nodezator, and it will likely only happen after other features like subgraphs are implement, so that we can properly take them into account in the solution. I can't provide an accurate ETA for such multi-paradigm concurrent/parallel graph execution for Nodezator. In fact, like any underfunded and understaffed open-source project, this is true for most features, even the smallest ones. I'm not complaining, though: the Indie Python project and Nodezator are super fun and I learn a lot, plus the few patrons we have are the best in the world. However, my work is far from being funded and is thus limited. This is just something to keep in mind. Despite not being able to provide an accurate ETA, with all this considered, and the complexity of the tasks we just discussed, it would not be honest of me to estimate anything less than a year or two, likely a bit more. However, who knows what the future may bring? With Nodezator userbase growing constantly (slowly but surely) and the constant efforts from me and the community to promote its usage, I hope to convince more individuals and organizations to help fund it, like other open-source projects manage to do. A possible more satisfactory workaround?However, there may indeed be some workarounds to this need of a multi-paradigm concurrent/parallel graph execution, as you cleverly pointed out. Not that it would replace the needed solution, as it is much needed, as we described above. However, it could prevent the GUI from freezing. As you said...
Indeed, if we manage to delegate the execution of the graph to another process running Python, we may be able to prevent the interface from locking and the OS to prompt for quitting or waiting. Of course, concurrency/parallelism in Python is never simple, and even this seemingly simple solution would still require careful consideration and design. It would be simpler than the actual needed solution, but still complex on its own. It would also have intrinsic limitations. For instance, Nodezator catches the data that travels through nodes during execution to provide extra services like displaying visualizations. This means that a workaround that delegates execution of the graph to another process would need to coordinate communication and data processing with that process between the execution of the nodes to ensure visualization data is also caught and properly conveyed in Nodezator interface during/after execution. In addition, the data synchronization between nodes would also need to consider more complex graph structures when subgraphs/group nodes are added. In other words, even the workaround is not without its challenges. And would still require much time, research and consideration. But I think overall it is a good start, and better than wait a couple years for the full needed solution. Nonetheless, I still won't be able to tackle it until subgraphs make into Nodezator. Keep in mind that subgraphs are already a complex feature per se (though most of the challenge is in regards to the interface, since the backend/data model should not be too challenging). Taking all this into consideration, even this workaround is something that I'll probably won't be able to tackle until the end of this year, given the many other features that should be introduced instead. Regarding your comment on the possible headless operation for Nodezator resulting from this suggestion of yours, I'm not sure such operation is that desirable at the moment, because in a way, everything Nodezator does is already headless in the sense that it works like a visual representation of Python (therefore allowing the graphs to be exported to pure Python code, which is already as headless as it could be). In other words, as I said in the Discord conversation you linked, a Nodezator graph can just be exported to Python and be executed like that with total control of the user and the ability to further edit. After all, it is just Python code, the more fundamental form possible. But I get your meaning. And this aspect of your idea of a headless feature does make sense. Some sort of graph-like, controlled execution, without a graphical interface could indeed be valuable, although as I said, probably not something that is that desirable right now, or perhaps I should say not as urgent. ConclusionIt is still regrettable that I can't offer an immediate solution for the problem, despite having a reason. I'm glad a hacky workaround like the one you are employing is possible at least. The truth is that Nodezator has many challenges to deal with and that can't be worked immediately not only because of limitations like time/people/funds and the large list of features awaiting implementation, but also because of the generalist/multi-purpose design of Nodezator that requires much more precision and care in the design of concurrent/parallel graph execution. That is, to ensure such graph execution doesn't harm/reduce Nodezator's capability to accurately represent and execute any Python callable, regardless of the paradigm for concurrency/parallelism adopted. In other words, we wanted power and flexibility, so we must deal with the cost of designing a system that can deal with all the resulting complexity. That is why I have deep respect for all other node editor projects (Python-related or not) despite the fact that most if not all gave up offering a language-centered generalist/multi-purpose environment. They reduced their scope or specialized in specific areas resulting in great reduction of their complexity and in many cases increased effectiveness when dealing the specialized problems. For a more ambitious app like Nodezator, the path is much more arduous, and it will take much more time to achieve even stuff which is basic in other apps, but I think the effort is worth it. Also, we are no doubt already reaping many of the benefits of our generalist/multi-purpose design. Nodezator can already be used with much power and flexibility for several computer/programming workflows and has the ability to combine all of them, despite the inconveniences and many things Nodezator still lack. Moreover, all this technical conversation and arguments presented are in no way meant to distract people from the fact that, despite being the creator and main contributor/maintainer of the app, I do have to improve myself as a developer before I can properly tackle these challenges. I do need to study and research more before I can properly design and implement such complex systems. This much must be said, in order to be as honest as possible with the Indie Python community and Nodezator userbase. Concurrent/parallel graph execution in Nodezator is just another challenge like the many others before it, but it requires more planning, time and effort than the previous ones, given the associated complexity. As always, let me know if you need anything else. This is a very complex feature associated with many complex topics. It can hardly be discussed exhaustively. There's always more to cover. Peace. |
Beta Was this translation helpful? Give feedback.
-
|
Totally forgot here about the scope of Nodezator, and it's clear why this is a hard problem to tackle. (I wrote some async TUIs in the past and it was a nightmare, never achieved truly seamless concurrency.) The workaround to tap into the pygame loop sounds cool, although that might be complex for what my projects require atm. What I could imagine for now is a clear UI indication before the freeze (so you know it started just by looking at the screen), and a clear indication for when it end (not just the text at the lower left). Some ideas:
|
Beta Was this translation helpful? Give feedback.
-
|
Great ideas. I think at least a dialog that shows the progress would make a lot of difference, similar to dialogs that are used when a program is downloading/processing several files. Something like "5 of 10 nodes executed...", etc. I didn't previously add anything like that because I had been mostly using it for relatively quick executions, so if a dialogue like that or other visual cue was used it would just appear very quick on the screen and disappear immediately, potentially leaving the user confused if the user didn't have the time to read the contents. However, as you pointed out, longer executions leave the user with no feedback at all, so it is better to have such kinds of indications anyway. I have been experiencing such problems myself since the end of last year when I started working with 3D and other more complex workflows that have longer execution times. I may add the dialogue like I mentioned, and then add the capability to dismiss the dialogue by pressing enter, as you have also been suggesting. I'll also add file-specific settings to allow the user fine control of this behaviour for each .ndz file they are working. This way users will be able to adopt different behaviours depending on the .ndz file they are working on. If it is a graph that takes too long to execute, then the user can turn the option to display a progress dialogue on, and if it is a quick execution and the user wants no dialog showing, the user can turn the option off. I may leave the option ON by default, since it is so handy, but I think it is important for users to be able to turn it off when desired because the user may want the screen free of dialogs, only showing the graph, at the end of an execution when the execution is quick enough and it updates visualizations shown in the graph that the users want to quickly inspect. This way the user can quickly edit-execute-inspect-repeat. I'll think about this carefully and comment back on it when I have a complete design, so you and others can opine. I'd actually like to add while implementing the other current planned changes. |
Beta Was this translation helpful? Give feedback.
-
Longer running operations (ML, downloads...) will lock up the interface with no indication of running.
Even worse, the OS will prompt for a Force Quit or Wait.
Possible solution is to cut up Nodezator into a core part (for running nodes) and the UI,
and run them on different threads/processes. (Not sure about GIL in Python...)
E1: (That might even allow headless operation in the future.)
E2: Slightly related Discord message
Beta Was this translation helpful? Give feedback.
All reactions