A rant about current custom nodes ecosystem

[wfjsw]

After spending an entire week getting out of the comfort zone of AUTOMATIC1111 extensions and trying to make some sense out of those popular custom nodes and reducing the amount of errors I came across, I finally decide that this is the time to document all the common bad design/implementation decisions regarding custom nodes, which will cause chaos and user misery all over the board.

While those are based on real user experience, all assertions here represent only my own opinion. Always take it with a grain of salt. This is not a completed list.

1. Non-Git and non-Folder compliant nodes

It is exactly a year since ComfyUI has become a thing now and there are still new nodes that requires copying a specific file to the folder, and ComfyUI-Manager is still providing support for such things, even when the node is hosted on GitHub.

ltdrdata/ComfyUI-Manager@a2c545b#diff-ca83200897149cebdfa21e3a4df3d00d4273d6812ba3adc21a2c1bb713dce484R3666-R3676

https://github.com/biegert/ComfyUI-CLIPSeg

Such nodes brings great difficulties when it comes to version upgrade and dependency management, and should be made compliant or removed ASAP.

2. Using common name for nodes

When developing nodes, please ensure that the name in NODE_CLASS_MAPPINGS is unique. One of the best way to do this is to prefix it with your name or project name or whatever you like, but please make it distinct so users won't have to guess which node they should install from a list where there is a ton of naming conflicts, while they are not necessarily compatible with each other.

3. Bundling nodes that has no relationship between

Please group your nodes and only bundle nodes in a same folder / repository if they have direct relationship, so users will not have to face all those conflicts that come wholesale when what they only want is a single node. Creating multiple repositories costs no money nor disk.

E.g. all those powerful node packs, such as:

https://github.com/shadowcz007/comfyui-mixlab-nodes
https://github.com/omar92/ComfyUI-QualityOfLifeSuit_Omar92
https://github.com/CYBERLOOM-INC/ComfyUI-nodes-hnmr
https://github.com/flyingshutter/As_ComfyUI_CustomNodes
https://github.com/AlekPet/ComfyUI_Custom_Nodes_AlekPet

4. Referencing external packages without listing them out in requirements

It is your responsibility to list out all requirements that you use in your nodes. This is not a thing that should be covered by either ComfyUI-Manager or ComfyUI itself. And the pip entries in index really should never be there in the first place.

https://github.com/ningxiaoxiao/comfyui-NDI/blob/e0c3d208623eb9c5388ec1613aad72d9eba4561c/__init__.py#L13

https://github.com/ltdrdata/ComfyUI-Manager/blob/8077765d47ffd9317f34f642c5fd87baadb9814b/custom-node-list.json#L3269

5. Trying to use pip in an unsupported way

It is important to know that the only officially supported way to invoke pip is through python -m pip. It is neither pip nor import pip. The pip or pip.exe could simply not be there and you risk installing some packages into a different environment rather than that you are running on.

Always combine sys.executable and -m pip.

https://github.com/shadowcz007/comfyui-mixlab-nodes/blob/3572368f16d13f0f496f2530757edc40eca8ec2e/__init__.py#L73-L74

https://github.com/shadowcz007/comfyui-mixlab-nodes/blob/3572368f16d13f0f496f2530757edc40eca8ec2e/install.bat#L16

6. Assuming users are all using python_embeded

I can no longer find any custom nodes doing this. Cool.

7. Specifying torch in the requirements

Why is my ComfyUI no longer able to use GPU? Why there is a CPU version of torch?

Just stop bothering with them.

https://github.com/biegert/ComfyUI-CLIPSeg/blob/7f38951269888407de45fb934958c30c27704fdb/requirements.txt#L9-L14
https://github.com/mav-rik/facerestore_cf/blob/2b5d727ed658e0b3feb14a620d67dad1b1bcb0ab/requirements.txt#L3-L4

8. Using unix utils

Honestly, why?

The only thing reliable in a Python environment is Python.

https://github.com/thecooltechguy/ComfyUI-Stable-Video-Diffusion/blob/2b2891c8046ade10d32f59d0178a500fd925de4c/prestartup_script.py#L20

9. Polluting global module tree

If you want to refer to SGM, that's great. But please please please vendor it inside your own namespace and don't let it out, especially when you are dealing with a trimmed version of it. It can be a big surprise for you and others.

https://github.com/chaojie/ComfyUI-MotionCtrl-SVD/blob/566fbdd8767ff293808e903288cde3bef8c6df68/nodes.py#L10-L12

10. Copying things out of the folder

We have WEB_DIRECTORY for ages now. Just don't do this. And atexit is not at all reliable.

https://github.com/AIGODLIKE/AIGODLIKE-ComfyUI-Translation/blob/eee96d109f59c75f519fcec973995cfef0398972/__init__.py#L126-L143

11. Doing weird things to ComfyUI internals

If you want to do something to the core ComfyUI functionality, put it into a separated node. I am not expecting my ComfyUI starts to listen to an HTTPS port when installing new nodes.

https://github.com/shadowcz007/comfyui-mixlab-nodes/blob/3572368f16d13f0f496f2530757edc40eca8ec2e/__init__.py#L383-L430

12. Use strict version specifiers

https://github.com/WASasquatch/was-node-suite-comfyui/blob/main/requirements.txt#L20

You ends up with this funny looking dependency resolution:

Extra reading: https://pip.pypa.io/en/stable/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020

13. Require onnxruntime

It is worth noting that, installing onnxruntime on top of onnxruntime-gpu will make the runtime lose the GPU acceleration functionality.

https://github.com/pythongosssss/ComfyUI-WD14-Tagger/blob/6b47b34cd61086215a3247c92884f733cd1e58b9/requirements.txt#L1

14. Meaningless folder / repository name

It creates madness when trying to figure out what the custom node is for from the folder name.

A good name should precisely describe the main functionality of the node. If you find it is hard to come up with such name, then you have a problem with section 3.

https://github.com/ltdrdata/ComfyUI-Impact-Pack
https://github.com/shadowcz007/comfyui-mixlab-nodes
https://github.com/WASasquatch/was-node-suite-comfyui
https://github.com/FizzleDorf/ComfyUI_FizzNodes
https://github.com/rgthree/rgthree-comfy
https://github.com/AlekPet/ComfyUI_Custom_Nodes_AlekPet
https://github.com/blepping/ComfyUI-bleh

Some good examples:

https://github.com/WASasquatch/FreeU_Advanced
https://github.com/Kaharos94/ComfyUI-Saveaswebp
https://github.com/city96/SD-Latent-Upscaler

15. Wrong dependency names in requirements.txt

This seems to be far more widespread than I originally think. This is harmful because it defeats any reasonable dependency analysis, and this is dangerous because any one can register virus under these unoccupied names and soon your node will help the baddies just like how LLMVISION did its thing.

https://github.com/chrisfreilich/virtuoso-nodes/blob/c1c23c07b50f16d669fd8cabce96970c35c2b583/requirements.txt#L6

https://github.com/jags111/ComfyUI_Jags_VectorMagic/blob/8c6cd6b3f5800d9c67e2a2b0978cec5232b2b8ed/requirements.txt#L11-L12

To be continued...

[ltdrdata]

This is a good guideline. Many developers creating custom nodes often start without prior experience in Python, myself included.
And without such a guide, various issues can arise as developers rely on their individual experiences while creating custom nodes.

Even with some experience now, I'm currently struggling to correct the buttons I initially misaligned.

[blepping]

Bundling nodes that has no relationship between

Please group your nodes and only bundle nodes in a same folder / repository if they have direct relationship, so users will not have to face all those conflicts that come wholesale when what they only want is a single node. Creating multiple repositories costs no money nor disk.

it may not cost money or disk but it can be expensive in terms of time. from a development perspective, it's much easier to have everything in one repo since convenience functions can be reused, infrastructure can be reused, etc.

also, even with most of my installed extensions basically being node packs i still have a lot of extensions installed. if every node was split out into a separate extension, it would be overwhelming to manage from a user perspective as well.

a better solution is to make one's extensions play nicely with other extensions. of course there are definitely cases for splitting stuff up logically, but it's not always possible or convenient and a lot of people just build collections of nodes that they think are useful and don't necessarily have a direct relationship. splitting those up would be impractical from both a development and a user perspective.

[wfjsw]

my point was managing them individually can easily get impractical. just for example, i personally install nodes (in practice, currently most are node packs) that seem like they may be useful. i have roughly 100 ComfyUI extensions installed. i don't know what the average number of nodes is for each, most have at least 3-4 nodes in them, some have 20+.

Looking at the numbers I have on my installation, I would say that those extensions that have less than 50 nodes are not subject to section 3 and 14.

[Counter] ambw_comfyui: 1 nodes.
[Counter] cd-tuner_negpip-ComfyUI: 2 nodes.
[Counter] ComfyUI-AnimateDiff-Evolved: 46 nodes.
[Counter] ComfyUI-Custom-Scripts: 12 nodes.
[Counter] ComfyUI-Frame-Interpolation: 13 nodes.
[Counter] ComfyUI-Logic: 7 nodes.
[Counter] ComfyUI-Loopchain: 8 nodes.
[Counter] ComfyUI-Manager: 0 nodes.
[Counter] ComfyUI-MotionDiff: 10 nodes.
[Counter] ComfyUI-Video-Matting: 2 nodes.
[Counter] ComfyUI-WD14-Tagger: 1 nodes.
[Counter] ComfyUI_ADV_CLIP_emb: 4 nodes.
[Counter] comfyui_controlnet_aux: 48 nodes.
[Counter] ComfyUI_Custom_Nodes_AlekPet: 9 nodes.
[Counter] ComfyUI_Cutoff: 4 nodes.
[Counter] ComfyUI_experiments: 8 nodes.
[Counter] ComfyUI_IPAdapter_plus: 12 nodes.
[Counter] comfyui_segment_anything: 5 nodes.
[Counter] facerestore_cf: 3 nodes.
[Counter] FreeU_Advanced: 2 nodes.
[Counter] LCMSampler-ComfyUI: 2 nodes.
[Counter] stability-ComfyUI-nodes: 3 nodes.
[Counter] StableZero123-comfyui: 2 nodes.

But those who violates them, they violate both of them:

[Counter] Comfyui-Toolbox: 3 nodes.
[Counter] ComfyUI-Impact-Pack: 165 nodes.
[Counter] ComfyUI-Inspire-Pack: 76 nodes.
[Counter] comfyui-mixlab-nodes: 61 nodes.
[Counter] efficiency-nodes-comfyui: 40 nodes.
[Counter] was-node-suite-comfyui: 212 nodes.
[Counter] WAS_Extras: 9 nodes.

[blepping]

Looking at the numbers I have on my installation, I would say that those extensions that have less than 50 nodes are not subject to section 3 and 14.

the problem is those collections didn't instantly start out with 50+ nodes, they gradually added features and probably a lot of those features were related to existing features. i.e. the pack includes something for face detailing, then later on it adds more nodes to support that feature, expand its functionality, etc.

you can't easily split it up at that point, because that will break all existing workflows.

also don't forget that the driving force behind probably the vast majority of open source projects is people scratching their own itch whether it's adding features or making a new thing because existing stuff doesn't work the way they want. then they think "hey, this might be useful for other people too" so they release it. i think it's pretty rare for people to just 100% altruistically start developing something they personally don't have a use for or intend to use.

[wfjsw]

the problem is those collections didn't instantly start out with 50+ nodes, they gradually added features and probably a lot of those features were related to existing features. i.e. the pack includes something for face detailing, then later on it adds more nodes to support that feature, expand its functionality, etc.

you can't easily split it up at that point, because that will break all existing workflows.

I am totally fine with a pack called ComfyUI-FaceDetailing to include 50 nodes for different face detailing mode. However, the majority of the node pack businesses are the ones that is called ComfyUI-Whoever-Pack and starts to pile in totally irrelevant nodes, for example, in a pack there are some model merging things, some latent IO things, some text translation things, and even some ChatGPT things.

This makes 0 sense.

[JerryOrbachJr]

@blepping im still fundamentally not understanding the context you are assuming. I'm assuming the author's stated conditions that nodes that have required dependencies on each other would be bundled together, and in that context I'm unclear what you mean by

i think you are seriously underestimating the difficulty. lots of nodes depend on other nodes, like pipes, loaders, etc. you install a node that looks interesting today and then try to use it a while later: then you have to try to track down and install the dependencies manually.

Regarding deprecating nodes, I think you are pointing out a good example of how the current ecosystem can make things easier in the short term but can make everything harder, both for authors and end users, in the long term. The Manager already has functionality to "fix" broken workflows caused by missing nodes - it tells you what the missing node is and allows a simple click to install it. If the practice was in place that authors would deprecate nodes and replace the contents of the node folder with a "deprecated.yaml" file which would point to the replacement node (and could have other useful info) the Manager could be updated to automate that process. But the custom node ecosystem is such a mess not because of author laziness (sorry if I gave that impression at all) but there are so few agreed-upon standards that authors can use to speed development (by using best practice templates for example) and because integral pieces like the manager have so little to consistently hook into.

also if it's split up by category, you still are going to end up with custom seed node #438739 if you wanted anything in the category of random stuff, noise, etc or however it was organized by that particular extension author.

I'm not understanding why under the author's proposed system things would be split up into categories like "random stuff." Am I being overly reductive by saying this sounds to me like "if people ignore these guidelines then these guidelines won't be useful."?

or why not donate your time to try to fix the problem?

You are absolutely right on this, and I will take this suggestion to heart and put more effort into contributing code to the ecosystem instead of commentary. I would just ultimately like to do so in a less chaotic and more organized and standardized environment so that my (and all custom node authors') work would be more efficient (i.e less work, more impact), which I was hoping to encourage in supporting the author's ideas.

[blepping]

and starts to pile in totally irrelevant nodes, for example, in a pack there are some model merging things, some latent IO things, some text translation things, and even some ChatGPT things. This makes 0 sense.

i mean, it makes sense for two (in my opinion very likely) possibilities:

1. i wanted this stuff, so i wrote it. here it is in the hopes that other people find it useful.
2. here are better versions a, b, c, [...] z nodes collected all together in one convenient place so you don't have to install a bunch of extensions.

but of those lines of thinking don't seem that unreasonable to me. like "efficiency-nodes-comfyui" basically says it's the second one right in the name.

---

im still fundamentally not understanding the context you are assuming.

i don't really know how to answer this part. you're assuming i'm assuming something but i don't know what that actually is. it would help if you quoted the part of my post you were referring to.

If the practice was in place that authors would deprecate nodes and replace the contents of the node folder with a "deprecated.yaml" file

sure, that's a good idea and there's a good chance people would use it if it was available. that functionality doesn't exist currently though, so it's depending on someone taking the time to add it to the manager.

but there are so few agreed-upon standards

i think it comes down to what i mentioned in before: i think people mostly create open source to scratch their own itch. also most people don't really find administrative stuff very interesting. i made some nodes because i wanted those features. i didn't sit down and say "well, i've decided to allocate some time to providing helpful nodes for the community. now what approach would be most efficient?" and i think most of open source is similar.

I'm not understanding why under the author's proposed system things would be split up into categories like "random stuff."

unfortunately i'm not understand your not understanding. there aren't that many ways to split things up. there's by logical category (split into categories like "random stuff"). just to be clear, by "random stuff" there i didn't mean miscellaneous things without order, i meant tools/functionality related to random numbers like seed, things that use it like noise, etc. maybe that's where the misunderstanding is? anyway, other than that you could organize stuff by what nodes were strictly depend on each other, or you could just split everything up regardless of other factors.

You are absolutely right on this, and I will take this suggestion to heart and put more effort into contributing code to the ecosystem instead of commentary.

👍 maybe it was still to harsh/blunt but what i was getting at was more like "put more effort into contributing code instead of saying what other people should be doing" not a general problem with commentary. i certainly don't think there's any kind of obligation to contribute open source stuff to... anything.