Indie Python
Indie Python in 2025 #6
Replies: 1 comment
-
|
Excellent update, @KennedyRichard! Static typingOf course, I'm excited about adding static typing to Nodezator, especially because I'm currently learning Haskell. I truly believe static typing would benefit Nodezator greatly. However, have you considered supporting the following:
from typing import Callable, TypeVar
T = TypeVar('T')
U = TypeVar('U')
def map(f: Callable[[T], U], lst: list[T]) -> list[U]:
return [f(x) for x in lst]
from typing import Protocol
class Show(Protocol):
def show(self) -> str:
pass
def show_all(lst: list[Show]) -> list[str]:
return [x.show() for x in lst]
class Animal:
def speak(self) -> str:
return "I am an animal"
class Dog(Animal):
def speak(self) -> str:
return "Woof!"
class Cat(Animal):
def speak(self) -> str:
return "Meow!"
def speak_all(lst: list[Animal]) -> list[str]:
return [x.speak() for x in lst]In other words, unless you have a PhD in type theory, you probably shouldn't try to support these constructs yourself! :) I hope you're planning to leverage the SubgraphsAlso, are you still planning to add subgraphs? I've been thinking about how to represent subgraphs lately and I came up with this in Graphviz:
Notice how the Again, this is a fantastic update. Nodezator has come so far! |
Beta Was this translation helpful? Give feedback.
-
Hello, everyone,
This discussion is meant to be a big announcement of what's planned for 2025 and the few years to come for the Indie Python project, as well as the reasons behind it.
As you all know the Indie Python project aims to promote fun and learning by publishing and maintaining open-source Python apps, games and related content. The flagship of our project is the Nodezator node editor which is maintained regularly and gets a couple big releases every year, but we also intend to finish and publish the Bionic Blue game, which currently is just a prototype/MVP. Other smaller projects are intended to be released as well in the future, like the tools used to make the game, like its level editor and pixel art/animation editor.
TL;DR (too long; didn't read)
I'd like to start with the announcements in this first brief section.
First of all, I just updated the Indie Python project's goal! The Indie Python project now also aims to provide value to people (not just fun and learning):
The Indie Python project aims to promote and provide fun, learning and value by publishing and maintaining open-source apps, games and content
Beginning in 2025, for the Indie Python project we'll...
In the next few sections I'll discuss each of these announcements in detail. As always, all feedback is welcome.
Why an app maker?
The Indie Python project creates and releases software (apps & games) dedicated to the public domain and completely free-of-charge. Even our patrons, people who donate regularly to the project, don't have access to any privileges or VIP services, despite donating monthly to the project. At most, they have priority when troubleshooting issues, but even this isn't much of a privilege per se since as you can see in both Discord and GitHub discussions I go to great lengths to ensure every and each person seeking help gets the help needed or at least a lengthy explanation as to why that thing isn't available yet.
In other words, donating to the project is a matter of conscience and a way to selflessly help support the project, acknowledging its importance to the Python and open-source communities. As any open-source project, specially one maintained mostly by a single person (me, Kennedy), the Indie Python project is always looking for new opportunities to raise funds to cover all development and operational costs.
This past 02 years since launching the Indie Python project I've been discussing back and forth with Mr. Adams (@WillAdams), the very first patron of the project that's been donating since day 1, about a lot of feature ideas for Nodezator. Many of such features included things like the ability to include custom interfaces, tools and shapes with different abilities to Nodezator, practically turning it into the user's own playground of ideas.
In addition to that, after reading much of his content online, I discovered he always yearned for a software tool similar to HyperCard. For those who don't know, HyperCard was a very popular software application that allowed non-technical users to create their own applications. It might have been one of the most, if not the most, successful software project as far as allowing people to create their own apps is concerned (perhaps along with Scratch and similar projects? Maybe spreadsheets too, in a sense).
At the very end of September this year I then reached him with a proposal to create a new app maker project just like that, with the condition that he increased his patreon donations accordingly in order to justify the extra effort and time put into this new project. Thankfully, he not only approved the idea but also agreed to increase his donations for at least a year, no strings attached.
The no strings attached part is actually very important for 02 reasons. First, because it reveals a deep understanding of the hardships of the software development process. And second, because it acknowledges the importance of open-source initiatives like the Indie Python project, that need time, effort and funds to grow. In other words, this is not contracted work with set deliveries and deadlines, but actually volunteer work whose value inspire people to donate selflessly.
I'm very glad for all of this, because as I said in emails exchanged between me and Mr. Adams, I have skills, experience and knowledge compatible with the requirements. It is simply something that intrigues and interests me. In summary, I wanted this opportunity to create a new project that may contribute to the Indie Python project's visibility as well as provide me with extra insight in ways to improve Nodezator in the future.
Below are a few interesting points/facts regarding the project:
Why Nodezator in different platforms or for different purposes
There is so much I envision for the future of Nodezator. Since before its release, when I was still developing and bringing together each of its different subsystems/services, I already thought often about how useful it would be to the Python ecosystem, about how many interesting things people could do with it.
I still have a long way to go in that regard, though. That is, in showing people the usefulness and advantages of node-based programming. And not only that, but also teaching people to use node-based programming in moderation, i. e., only for the purposes for which it is suited. The reason is that some node editor projects try to shove their tools down people throats as silver bullets that can replace regular text-based programming and be used for any purpose. I don't think that's true. That misbelief is detrimental both to people believing it and to node-based programming in general. That's cause the misuse of node-based programming lead to poor results and makes people abandon node-based programming altogether.
As many know, I still have a lot of questions, but thankfully I also have opportunities to research and test node-based programming suitability for many different kinds of tasks. Of course, my work as an open-source maintainer means I spend more time developing and maintaining software than doing research. This is something that I'm trying to change, because research is much important to my work as a developer and, as a consequence, to the quality of our software as well. Here's a brief thread on a few experiments I made with node-based programming in 2023: https://bsky.app/profile/kennedyrichard.com/post/3ldlfantczc2d
With so much potential still unexplored, I want Nodezator to be used by many people, and then I want to collect their feedback and use it to improve Nodezator even more so it can ultimately become a tool that many people can find useful and inspiring. Much like Blender 3D, which is the project that inspired me to create Nodezator (I always wanted a version of Blender's node editor that I could use for general computing).
However, 02 years after its initial release and after a few big updates and many new features, I have a much clearer picture of what Nodezator is missing in order to become even more useful to people.
As someone who loves pygame(-ce) and gamedev in general, I thought pygame was the best platform possible for Nodezator. And guess what, I still think Nodezator in pygame is awesome and want to keep working with the library forever. However, as much as there are innumerable things people can achieve with Nodezator + pygame, more so than they can even conceive, there are also innumerable things people can achieve with Nodezator + PySide/PyQt, or with Nodezator + Tkinter, or with Nodezator + dearpygui, and so on. Not a single one of these libs/frameworks are better than the other. They simply offer different advantages.
Because of that, I want to bring Nodezator to many platforms and also allow people to embed it in their apps in such platforms.
Let me try to help you perceive why this would enrich the Python ecosystem even more. This is the first node shown as an example in Nodezator's readme file, along with the code which generates it automatically:
Look how pure that function definition is. Apart from Python's math module, that definition doesn't require any other import. Not from Nodezator, nor from any other library. That's cause Nodezator strives not to pollute a node's definition, to keep it as pure as possible. When it comes to defining nodes, Nodezator actually works more like a protocol on how to indicate the callable to use for the node (and other useful info as well).
There's no reason or excuse to keep such useful tool from other platforms nor from other people's applications. Even less so now that the goal of the Indie Python project includes providing value to people. We need to deliver the value provided by Nodezator to other platforms/frameworks and apps (as a widget/interface). And we'll begin by making a new version of Nodezator for the Qt framework and to provide Nodezator as a widget/interface to PyQt/PySide and pygame applications.
Changes in Nodezator (type safety, new data model and more)
Back when Nodezator was created
As some of you know, I only picked Python programming seriously at the end of 2017 and then started making Nodezator at the beginning of 2019. At the time, despite my little experience with Python, regarding Nodezator's design, I like to think I made many great decisions that made Nodezator what it is today.
However, given not only that same lack of experience in Python, but also the lack of experience in the design of node editor interfaces as well as the lack of knowledge/feedback on the many challenges of node-based programming, there were also many decisions that I want changed/removed from future versions of Nodezator.
Which is a natural thing. We do, we learn, we improve. That's the reason software (usually) keeps getting new versions.
As a result, there are many things I want to change in Nodezator. In fact, too many to fit in this announcement, so I decided to only discuss a bit about type safety and the data model for now.
Type safety in Nodezator (and actually a lot more stuff as well)
First of all, I'd like to make Nodezator type safe, that is, make it so the connections can only be established between compatible types. Of course, people who want will still be able to keep using Nodezator without type safety, as it is a valid and useful way of programming that should be free from stigma. To each their own.
One of the reasons it isn't so today is that back when I created it (2019), type safety wasn't even mentioned in most Python learning materials. It is a relatively new concern in Python programming. Back then annotations/type hints we just that, hints about the expected types of function arguments. Some libs/apps just used them to store metadata about an argument. It was how I decided to use them in Nodezator.
In Nodezator the annotation means "the kind of data to be edited" and the default value means "the initial value for the widget to hold". The annotation, thus, is used to define the widget used on the node for that parameter and its default value is the initial value stored in that widget.
For instance, check this simple node definition and its resulting node:
This way of using annotations is actually fine in a lot of cases because, like shown in the previous code and respective node, it clearly conveys both the kind of widget to be used (a checkbutton) and properly conveys the types used for type checker software (and humans as well).
However, there are also many cases where this "type hints as metadata" makes it impossible for type checkers to identify the expected types. That is, widgets which are more complex usually require a lot more info to be conveyed through the annotations, in which case we can't specificy only the type in the annotation, but a whole lot more data is needed, which we deliver via a dictionary. For instance, this is how we define a widget for a parameter that expects a string representing an image path (this widget in particular is useful to preview the image the path points to):
Here's what the node looks like when the path stored in the widget points to an image file or not:
In other words, the
image_pathparameter has{'widget_name' : 'image_preview', 'type': str}as its annotation, which is clear enough to be understood by humans, but cannot be understood by type checkers.The thing is, while sometimes the type can be used to precisely infer the right widget to be used, a lot of times this isn't true. The
booltype, for instance, only has 02 possible values, eitherFalseorTrue. A checkbutton, which only has two possible configurations, marked or unmarked, satisfy all requirements of this kind of data.The
strtype, on the other hand, may represent many things. It may represent simple one-line text, in which case a simple text input widget can be used, or it can represent multi-line text, in which case a text display widget is more appropriate (a widget similar to a textarea in HTML). Astrcan even represent much more than that. As we just demonstrated, it can represent a path to an image file (and any other file or a folder). It can even represent an image itself (see the PNM format and its subformats).There is still another problem with that usage of a dictionary as an annotation to define the
image_pathin theget_image_sizefunction. Just compare that function definion with the previous function definition for theinvert_boolfunction: while theinvert_boolfunction only contains data about the function itself and the task it performs, theget_image_sizefunction contains the extra metadata to define the widget for theimage_pathparameter, and such metadata is of interest only to Nodezator.In other words, annotations like
image_path : {'widget_name' : 'image_preview', 'type': str} = '.'do pollute the function. And as I said before, Nodezator strives not to pollute the function. That is not a question of personal preference. There are many great reasons for that.First and foremost, unless a framework/interface using your function has a good reason for making your import things and/or change your function, changing your function to suit another interface just increases noise1 and infoxication and decreases its reusability. In other words, extra information in a function or other callable is not only needless, but also harmful, decreasing the value of the function and making it potentially confusing.
Second, adding foreign or extra elements in a function or other callable might end up increasing its complexity somehow. The simpler the functions, the more easy it is to combine/compose them into powerful graphs. I like to compare nodes with Lego® pieces: the simpler/smaller the Lego® pieces, the more freedom of building one has. For graphs it is just like that as well. It becomes more easy to compose them into many different graphs.
Python is a multi-paradigm programming language (though mostly object-oriented), but one of the most powerful paradigms in terms of both power and flexibility is FP (functional programming), and it is one that works particularly well with node editing. That's why Nodezator's user manual demonstrates/teaches many FP tools that can be used in Nodezator. Just like in Lego®, FP also encourages simpler/pure functions.
I'm not talking about simplicity as the opposite of complexity here. Of course a node definition can still be as big as needed (just like the more specialized, thematic, bigger Lego® pieces). The node source may even consist of multiple modules. However, it should only contain code and resources for the sake of its goal, not foreign/extra code/data that doesn't belong in its logic.
The third and final reason is for the sake of concurrency/parallelism. The less state/side effects an operation has, the easier it is to run it concurrently or in parallel. As such, the more pure the function definitions used to define nodes are, the easier it is to execute them non-sequentially.
So, how do I intend to improve how Nodezator uses annotations in order to make it more type safe? There are actually many improvements/changes that need to be made, so I'll mention only a few.
One of the measures to be taken is to deprecate the usage of dictionaries in the parameter annotations to define widgets, using annotation only as they are supposed to be used: to indicate the type(s) accepted by that parameter. As is the case with the
invert_boolfunction, using type hints like that still allows Nodezator to pick the appropriate widget.However, we still need the user to provide extra data when it comes to specifying more complex widgets to be used, as we saw in the case of the
image_pathparameter for theget_image_sizefunction, which needed a dict wherein to specify the name of the widget. Other widgets require even more data, like the option menu widget, which needs a list of the possible values to be used in it. When we design an API/interface for people to create and distribute their own custom widgets, the number of widgets available will be even higher, requiring even more data.In other words, we can't avoid asking the user to provide extra data about the widget(s) they desire. However, another measure I'll take is to move this info outside the function definition (or whichever callable is used), that is, it won't be in the parameter annotation anymore. The annotation will be used only to define the type(s) accepted. Instead, the user will specify the widgets to be used in a dedicated variable or collection outside the function. Here's an example of how the
get_image_sizewould be, though it is merely illustrative (the final format used wasn't designed/defined yet):In other words, now the function is pure. It only contains logic for what it does, no extra code/data. Such extra data is now outside the function.
Remember what I said in a previous section about how Nodezator works more like a protocol and how I want it to be available in other platforms? Well, by doing something like we did in the code block above, the function definition could contain specifications for how to turn the function into a node in different platforms/apps, even other apps that are not forks/versions of Nodezator. Something like this:
I know this makes the node definition even larger, but look at what is important: the function representing the node is super simple and pure and can properly be read by a type checker. And your function definition is free to be used in whichever platforms you desire, as long as it is compatible with one of the given specifications.
Another advantage of having the allowed type(s) specified in the annotations is that it give us freedom to use any compatible widgets we want. We don't need to limit ourselves to the widgets in the specifications anymore. We just search the function signature for the type(s) accepted by that parameter and we can then switch to any compatible widget.
For instance, a parameter tha accepts a tuple of integers representing the RGB (red, green and blue) values of a solid color could use a color button widget where the user would click the button and pick a color from a color wheel or could switch the widget to a simple Python literal input where the user could simply type the values manually if they find it more convenient at the time.
Widgets are foreign elements that are part of the node editing interface, that is, the GUI. They are not part of the function definition. The function definition only gives us the type for the parameter, but we should be free to pick whichever widget we want, as long as it is compatible with the annotation.
There are many, many changes like the ones I just presented here that I want to make in order to improve Nodezator, make it more type safe and the node definitions more pure and reusable. However, as I said at the beginning of this section I can't present everything here, lest this simple announcement would become even larger. I'll present, discuss and gather feedback for each change over time, as the time to implement them gets close. Or maybe I'll compile all of them in a single large discussion and publish it sooner. I didn't decide yet.
For now, let's go to the next section.
New data model for Nodezator
As a project that aims to be a visual equivalent of the Python programming language, Nodezator uses a plain text format (which is actually a Python dictionary) and also exports its graphs directly to Python code.
However, node editors are widely regarded as multimedia applications. It is no coincidence that this kind of interface is used a lot for media editing in many different fields, like 3D design, video production, etc. Striving to make Nodezator more close to the programming language, I ended up making design decisions that unintentionally and needlessly made it less easy to use Nodezator as a multimedia application. Namely, the Nodezator file (the .ndz file) doesn't store media files, data files or code files. As I said, it is just plain text that only references external data/media/code as needed.
Because of that, Nodezator files that reference data/media/code can't be shared as-is, but rather need the corresponding files to work properly. And that's not the only problem: references to other files/folders (paths) in the .ndz file are all stored as absolute paths. That is, when opening an .ndz file in another computer, even when all referenced files/folders are present, one still needs to fix the paths to point the appropriate new absolute locations.
This is why one needs both the .ndz files and node packs used in order to open the .ndz file.
Now, don't get me wrong, this is not bad per se: what is bad is the fact that this is the only option. Allow me to explain: being able to reference external things is powerful, it is how programming works. For instance, when you write a script using the Pillow or matplotlib libraries, instead of distributing copies of such libraries, you only distribute your script, since people can get those libraries from a central place where a compatible copy will be delivered to them. This avoid a lot of data/code duplication.
However, if a script, or in our case a node graph, is small enough (or even when it is big, if its scope is small), embedding the node packs and media files associated wouldn't be a problem and in fact would be very convenient, as you'd be able to share the .ndz file as-is and the person on the receiving end would be able to open it directly without having to setup anything.
In other words, we must keep Nodezator's ability to reference external resources, but also expand its capabilities so it can store resources as well. For that, I intend to change Nodezator's format to use a .zip file representing a folder with all related data/media/code. Many other software project actually already use data models like this one or similar ones: MS Word files, .blend files, Godot projects, etc.
The ability to reference external resources must also be expanded so relative references (paths) can be used. This way, the user can also decide whether to store the external resource within the .ndz file or simply share the resources along inside the same folder so the relative references within the .ndz file can still locate those resources regardless of their absolute location.
In fact, all of these abilities must be independent of one another. That is, a user must be able to include both absolute and relative references to external resources in the same .ndz file, and even store resources in it as well, all at the same time.
This will make "playing/experimenting" with Nodezator even more straightforward. It will make Nodezator more "plug and play". Currently, Nodezator doesn't come with any specific code/media/data or other kind of resources. All of that must be provided by the user, that is expected to be an intermediate Python user. Because of that, users who never used Nodezator in their lives, when opening the app for the first time, are greeted with the cold lifeless grey interface that hides a world of possibilities below.
Experienced programmers won't be intimidated by that and will be able to progress past this first underwhelming experience to find the actual treasure that Nodezator provides. Less experienced programmers though don't know where to begin. And since other node editor are usually hyperfocused in specific fields/tasks and offer many built-in multimedia capabilities, they become frustrated because in Nodezator they can't do much right away.
Although Nodezator is supposed to be a visual representation of Python, many people using it don't expect it to act like the Python interpreter, that is powerful but only do what it is told to, and one must write the code for it to interpret and run. People expect Nodezator to contain many colorful boxes and powerful effects/capabilities form the get-go.
As Nodezator creator, maybe my main shortcoming was not realizing this sooner. Again, being able to reference external resources is very powerful and must not only be present, but encouraged as well. However, we must also not forget people's expectations and mental model of node editors: that node editors are multimedia applications with many built-in capabilities.
Because of that, I also intend for Nodezator to never present an empty again screen again (under the splash screen). That is, just like Blender 3D and other software projects, I want Nodezator to always open with a valid simple project already loaded by default. Blender 3D launches with a basic scene loaded (a default file), containing a simple elements (cube, lamp, camera) that allow people to start experimenting/playing with the software right away (or if they are experienced users, just change the default file to a plain empty one if they desire, or simply delete the basic elements like many people do with the default cube in Blender).
In the same vein, I'd like Nodezator to present a basic graph on launch, where people can play with the nodes a bit and achieve interesting/fun results. In order to enable this, we must allow .ndz files to store resources/assets, so this graph can contain media/data/code, etc., as needed.
Finish and publish the Bionic Blue game
I'd like to finish and publish the Bionic Blue game in 2025. I'd also like the first level, including a boss battle, to be finished and released at the very beginning of 2025.
Despite all the potential of Nodezator, its many versions we are planning and the new app maker project, we must not forget the role of video games and game development in the Indie Python project. I was actually writing a comprehensive article about it in the form of a chapter in Nodezator's SDD.
The chapter is not finished yet, but it already presents many interesting points in favour of the usage of video games and game development in many areas and for many purposes. Gamedev drives creativity and innovation, helps in learning programming languages and also promotes socialization and the exchange of ideas. As mentioned in the chapter...
As an open-source maintainer, I'd like to both recommend and practice the inclusion of open-source gamedev projects as part of open-source projects. That's why I want the Indie Python project to always have a game project being developed along the other software at all times. That is, as long as one game project is finished and released, we'll create another (which might be a sequel of a previous one or not).
Despite not having any level ready, the Bionic Blue is already almost feature-complete. Its current release version is a MVP/demo/prototype version which already has the playable character perform most of the actions planned for it and is able to traverse the terrain, damage and defeat enemies, be damaged and die. It also has input recording and playing features, which only needs small adjustments to be fully usable. These features can be used to record and play playtest sessions to help improve the game.
In other words, what the game lacks most now is content (props, enemies and bosses) along with the corresponding behaviour as needed. Of course, in practice, the missing pieces may take more time than anticipated, but it is reassuring to know that the main features are already there and the MVP/demo/prototype already works.
Finally, I'd like to apologize for all the delays. Nodezator requires a lot of development time, user support, content production and management work, which leaves me with little to no time for the game. However, as I said here and in previous discussions, I want to give the game project within the Indie Python project the same priority the other projects have, which is something I'll strive for going forward.
End of the announcement
So, that's the end of the announcements. Thank you for your attention.
The Indie Python project produces and releases apps, games and content all free-of-charge and in the public domain. Please, consider becoming our patron for some time or making a one-time donation via one of the other available donation methods. All donation methods can be found here: https://indiepython.com/donate
As always, all feedback is appreciated.
I wish you a merry X/Christmas and a happy new year.
Peace
Footnotes
noise: unexplained or unexpected information in a sample that is not useful and that can be ignored (Cambridge dictionary) ↩
Beta Was this translation helpful? Give feedback.
All reactions