Rollback extra networks tree view

[kaalibro]

Description

#14588 is a great feature, but it has encountered several usability issues that have been noted by users.

The most common complaints include:

• Increased user effort required to select the desired card.
• Noticeable performance degradation when handling a large number of Extra Networks.
• Excessive space occupation within the UI.
• Issues with sorting functionality.

I propose implementing this feature as a built-in extension. This approach would provide users with the flexibility to enable or disable it as needed without impacting the main codebase.

This PR aims to rollback the changes related to the tree view, preserving all subsequent commits up to and including 58f7410. @Sj-Si agreed with this approach.

My suggestion is to merge this PR with the dev branch while creating a separate one specifically for the tree view.

Checklist:

• I have read contributing wiki page
• I have performed a self-review of my own code
• My code follows the style guidelines
• My code passes tests

[Sj-Si]

Sorry guys I guess I dropped the ball on this one. I am working on changes that will hopefully please everyone. But for now I think rolling it back will at least quell the storm a bit and give me time to work.

I'll also have to figure out how the whole built-in extensions thing works so it may be a while.

[light-and-ray]

Maybe instead of deleting everything you should have added option to bring back old view?

I propose implementing this feature as a built-in extension

It's impossible. You saw the code

[kaalibro]

Maybe instead of deleting everything you should have added option to bring back old view?

Many will be grateful to you if you implement this solution, while correcting all the shortcomings listed above and not creating new ones.

[light-and-ray]

You want this - you make a pr. I don't want. And I'm against of this kind of vandalistic pr

[w-e-w]

Noticeable performance degradation when handling a large number of Extra Networks.

possible related

• [Bug]: webui connection errored and reloads webui when switching tabs or searching loras or changing models, etc (like it gets stuck and resets after it has been a while) #15143 (comment)

[kaalibro]

possible related

• [Bug]: webui connection errored and reloads webui when switching tabs or searching loras or changing models, etc (like it gets stuck and resets after it has been a while) #15143 (comment)

the cursur gets stuck as the last thing u were hovering on, like if it was a button it will get stuck as a pointing hand

same thing

[light-and-ray]

Noticeable performance degradation when handling a large number of Extra Networks.

possible related

I had the same before the update

[light-and-ray]

May be related too

• load extra networks html on create_ui #14990

[light-and-ray]

But I still think vandalism is not a solution

[Sj-Si]

But I still think vandalism is not a solution

Neither do I however it was because of this change that some users are practically unable to function at this point. So either we do this or these people have to wait for an entire development cycle before they can use the app as they did before.

When I originally tested my code, I was testing with a couple hundred safetensors files not realizing that people out there have thousands of these things for some reason.

I'm still working on the laundry list of bugs that people brought to my attention so I don't know how long it will be before I can even put in a PR with the revised code. I wish more people used the dev branch so that I could have found out about these breaking bugs before it went to release. :(

[clayne]

But I still think vandalism is not a solution

How is this vandalism? The PR just looks like a revert - which isn't vandalism, and the OPs description of the reason why is completely reasonable. You may disagree with his assertion but that's a separate issue.

I personally use an extension to get rid of the tree view due to usability issues of its own but I'm not against rolling this back until it can be revisited with more use cases / lora counts in use.

[AUTOMATIC1111]

For now, let's rather fix issues and offer support for both. See if e0c9361 solves the performance degradation.

[kaalibro]

See if e0c9361 solves the performance degradation.

Not very :)

2024-03-08.10-14-19.mp4

[kaalibro]

For now, let's rather fix issues and offer support for both.

It's not difficult for me to create a fork, roll back locally, and await the update. However, wouldn't it be wise to roll back now so that other users, who may not be familiar with development, can work comfortably in the meantime?

however it was because of this change that some users are practically unable to function at this point. So either we do this or these people have to wait for an entire development cycle before they can use the app as they did before.

[AUTOMATIC1111]

try e0c9361 if you want to see if it improves the performance or not

[kaalibro]

try e0c9361 if you want to see if it improves the performance or not

I've already done

[AUTOMATIC1111]

Oh, right, I misread that as a different hash. Are you seeing no improvement at all? The string catenation thing was taking 30 seconds for me and was reduced to nearly zero.

[kaalibro]

Are you seeing no improvement at all?

I also assumed that it would help, but my performance remained the same. I even recorded the video on the third attempt, during which the page crashed completely once :)

Although the performance remained same as here, I didn't encounter that issues previous time. I deliberately didn't test without OBS to maintain consistent conditions.

UPD: Without OBS "Refresh" takes 34s (vs 38s on video) with Tree view and 18s with rolled back Buttons.

[AUTOMATIC1111]

a551a43 adds the option to have old dir views, set to True for now. If it's enabled, the create_tree_view_html function is not called, so try again and see performance problems are still there - then we'll know id they are in that function or somewhere else.

I actually can't reproduce performance problems. Granted, I don't actually have 8k loras on disk, I use a loop to add same loras many times.

[light-and-ray]

I have about 100 loras and I thought it's many 😐 OP, how much loras you have? If its actually about few thousands, so I think it's needed to write a real file manager 😆

[AUTOMATIC1111]

The guy in the issue has 8k, and since it worked well before, it should work well with this many after the feature.

[light-and-ray]

I agree, there need to be option for old view

I mean for tree view of such amout of loras there should be optimization tricks like in real file manager

[kaalibro]

how much loras you have?

My 10k "loras" are actually blank .safetensors files created for testing purposes. But some users do have several thousand of them — I have seen this repeatedly.

try again and see performance problems are still there

Tree View a551a43

Refresh w/o OBS — 16s
Refresh with OBS — 24s
Search by name — 10-15s
Folders switch — 3s
Sorting — alphabetical

treeview.mp4

Dirs d062345

Refresh w/o OBS — 13s
Refresh with OBS — 20s
Search by name — instantly
Folders switch — instantly
Sorting — natural (more intuitive)

dirs.mp4

[AUTOMATIC1111]

Pushed optimizations for sorting and filtering, should be as good now. I also made 10k empty safetensor files and am testing on those. Refresh with current dev takes about three seconds, with this PR's version - nine.

[Dasor92]

I just tried the new dev branch, love the buttons! but it broke some XL loras, it gives a "shape" missing error. Could you investigate on that? (for example: Anime Cold Night Style SDXL_LoRA_Pony Diffusion V6 XL)

XL.safetensors: AttributeError
Traceback (most recent call last):
File "D:\Programs\stable-diffusion-webui\extensions-builtin\Lora\networks.py", line 280, in load_networks
net = load_network(name, network_on_disk)
File "D:\Programs\stable-diffusion-webui\extensions-builtin\Lora\networks.py", line 219, in load_network
net_module = nettype.create_module(net, weights)
File "D:\Programs\stable-diffusion-webui\extensions-builtin\Lora\network_lora.py", line 11, in create_module
return NetworkModuleLora(net, weights)
File "D:\Programs\stable-diffusion-webui\extensions-builtin\Lora\network_lora.py", line 18, in init
super().init(net, weights)
File "D:\Programs\stable-diffusion-webui\extensions-builtin\Lora\network.py", line 150, in init
self.dora_mean_dim = tuple(i for i in range(len(self.shape)) if i != 1)
AttributeError: 'NetworkModuleLora' object has no attribute 'shape'

[AUTOMATIC1111]

self.dora_mean_dim = tuple(i for i in range(len(self.shape)) if i != 1)
AttributeError: 'NetworkModuleLora' object has no attribute 'shape'

@KohakuBlueleaf this is the code added yesterday

[kaalibro]

Pushed optimizations for sorting and filtering, should be as good now. I also made 10k empty safetensor files and am testing on those. Refresh with current dev takes about three seconds, with this PR's version - nine.

Tree View (current dev) 7d1368c
Refresh w/o OBS — 16-18s
Search by name — instantly
Folders switch — instantly
Sorting — alphabetical

Dirs (this PR) d062345
Refresh w/o OBS — 17-18s
Search by name — instantly
Folders switch — instantly
Sorting — natural (more intuitive)

Now I have the same refresh and filtering times for both branches. But "By path" sorting still works differently.

[Sj-Si]

I've nearly finished some updates that will massively improve performance across all extra network tabs. I've pretty much rewritten everything again but this time using Clusterize.js on both the tree view and the cards panes. Still working through some bugs at the moment but so far it is leagues faster.

The one tricky thing that I haven't figured out yet is how to just directly pass data from ui_extra_networks.py to a function in extraNetworks.js so if anyone has any idea how to do that I'd appreciate some help. So whenever ExtraNetworksPage.create_html() is called, I want to be able to manually pass some JSON data from the python app to a function in extraNetworks.js.

[AUTOMATIC1111]

<script> with var extraNetworksData={...} inside.

[KohakuBlueleaf]

I just tried the new dev branch, love the buttons! but it broke some XL loras, it gives a "shape" missing error. Could you investigate on that? (for example: Anime Cold Night Style SDXL_LoRA_Pony Diffusion V6 XL)

XL.safetensors: AttributeError Traceback (most recent call last): File "D:\Programs\stable-diffusion-webui\extensions-builtin\Lora\networks.py", line 280, in load_networks net = load_network(name, network_on_disk) File "D:\Programs\stable-diffusion-webui\extensions-builtin\Lora\networks.py", line 219, in load_network net_module = nettype.create_module(net, weights) File "D:\Programs\stable-diffusion-webui\extensions-builtin\Lora\network_lora.py", line 11, in create_module return NetworkModuleLora(net, weights) File "D:\Programs\stable-diffusion-webui\extensions-builtin\Lora\network_lora.py", line 18, in init super().init(net, weights) File "D:\Programs\stable-diffusion-webui\extensions-builtin\Lora\network.py", line 150, in init self.dora_mean_dim = tuple(i for i in range(len(self.shape)) if i != 1) AttributeError: 'NetworkModuleLora' object has no attribute 'shape'

Known issue caused by me

[light-and-ray]

I think it can be closed. New option to view selection is good

[AUTOMATIC1111]

natural sort order is in now

[kaalibro]

As for me, we can say that at the moment Tree View can be used 90% without pain :) Special thanks for the separate display of sorting options.

There are still a few bugs:

• Sorting "By path" remains alphabetical for Textual Inversion and Hypernetworks tabs.
• In "Tree" display mode, folders in the Checkpoints tab aren't displayed on the pane if the checkpoints are located in an external folder and loaded with the --ckpt-dir arg. The cards themselves are visible. With "Dirs" mode, folders buttons are displayed.
• In "Tree" display mode, hidden directories are shown on the pane regardless of the "Show hidden directories" option state. This behaves correctly in "Dirs" mode: when the option is disabled, the buttons of hidden directories aren't displayed.

Minor bugs:

• If the "Show hidden directories" option is disabled and you open any Extra Networks tab immediately after UI load, cards from hidden directories are still displayed. They disappear after updating the view in any way (Refresh, switching Extra Networks tabs, switching directories in "Dirs" mode, or searching by name). Also, hidden cards are not visible if you open Extra Networks a few seconds after UI load.
• "Copy path to clipboard" copies the path without backslashes.

Since this PR is not needed to fix these bugs, I'm closing it. Many thanks you all for participating!

[remixer-dec]

This is how tree view looks like on macos in firefox/safari