ExtraNetworks: Performance Updates and Improvements

[Sj-Si]

⚠️ THE FOLLOWING COMMENT IS THE MOST UP-TO-DATE STATUS OF THIS PR. START THERE: ⚠️

#15530 (comment)

⚠️ DESCRIPTION BELOW THIS POINT IS OUT OF DATE ⚠️

Description

This PR is a complete overhaul of the code that I previously wrote for the ExtraNetworksTreeView PR (#14588) in response to the negative feedback such as in #15164 and various issues that have been posted.

The goal of this PR is to improve performance and reduce resource use across ExtraNetworks tabs.

I have tested these changes with up to 100,000 models at a time. With this many models, the initial load is slow but as soon as the server fully generates the dataset it performs very well. Before, my browser would just crash if I even tried to load the page with more than 50k models. The real limiting factor here is the server (though it takes quite a bit to reach the limit). Right now, ui_extra_networks.py is generating the HTML for all of the files it discovers in any of the models directories. It then waits for requests from the client and sends only the requested items HTML back to the client. This way the client doesn't have massive amounts of DOM elements constantly loaded. So as long as the device running the server can load the generated HTML for every single item in the models folders then it should be fine. I don't see this being an issue until some psycho comes along with 10 million models but I don't really think we should be pandering to such a niche audience. Even then, I think other things will crash before this new code does.

Sorry ahead of time... this is a big one.

The following has changed:

• Implemented a modified version of Clusterize.js to allow for an asynchronous data loader.
  • This allows us to offload generating and storing HTML strings to the server instead of the client.
  • HTML strings for tree/card elements are generated in the python server and only fetched by the client via JavaScript when they are needed. Fetching is done in smaller chunks to reduce overhead.
  • Set up a pseudo paging system to use as a cache for fetched HTML elements. This way we can reduce the number of times we fetch data from the server.
• Reduced DOM complexity by minimizing the amount of nested elements in the tree/cards views.
• Created all new SVG icons for the ExtraNetworks tab controls.
  • Accidentally went on a tangent and ended up learning how SVGs work.
  • I designed all of these SVGs so we don't have to worry about licensing crap.
    • I'm not a graphics artist in any way so feel free to change these designs as needed or draw up a sample and I'll try to update it.
• Created a utils.js file and added a lot of helpful utility functions that could probably be used elsewhere if needed.
• Tab now allows use of both the tree and directory views at same time.
  • Selecting a filter button in one view selects the corresponding button in the other view.
• Fixed various bugs that went undiscovered in my previous implementation.
• Added a custom event that fires whenever a resizeHandle.js handle is double clicked.
  • This allowed me to implement an auto-resize function on the tree view whenever the resize handle is double clicked. This automatically sets the width of the tree view to the width of the widest row in the list.
• Added an option to only show directories in the tree view per some user's request.
• Updated CSS to use variables instead of hardcoded values.
  • This way these changes should be way more compatible with custom themes.

UI Changes

Below is an example of what the new format looks like:

Controls

The controls have been grouped with a header to hopefully make their intent more obvious.

Directory View

Directory view buttons have been styled a bit better and now the selected directory will be highlighted.

Tree View

Slightly modified the appearance of the tree view and simplified the code that generates it. Also directories in the tree view are no longer expanded when the item itself is clicked. Now the user must click the chevron on a directory in order to expand it.

Checklist:

• I have read contributing wiki page
• I have performed a self-review of my own code
• My code follows the style guidelines
• My code passes tests

  ⚠️ Can't run tests with my environment however the tests in CI did pass.

Additional Notes

Someone suggested that we use emojis for the controls in the extra networks tabs. Personally I hate emojis and they look disgusting. On top of that, I couldn't find any that really fit the purpose of each of the controls. If someone has a theme (like lobo or whatever) then they can handle replacing these SVGs if they want. The SVGs are all easy to query so they can handle a find/replace if they want.

One thing that I couldn't figure out is how to manually trigger the gradio loading icon within the extra networks pane (see below):

This loading page appears whenever the page is reloaded or refreshed but I would have liked to have been able to manually trigger it to show up and stay there until the data is fully loaded but I couldn't figure out where this loading pane was even coming from. I'd really like to be able to put one in the Tree View and one in the Cards View. Then I would trigger each manually whenever I need to load more data.

Also at some point someone requested that clicking one of the directory view buttons to filter cards should only show direct children of that directory and not any of the nested directory content. What was the consensus on this? Should that be the behavior or do we prefer to show all nested items? The current behavior is the same as it always has been and it is a pretty simple implementation where clicking a button simply adds that path to the search box and updates the filter based on the search box text. Changing the behavior of clicking directory buttons would definitely require a bit of redesign and I think this change has already had enough scope creep as is.

Needs Testing

I would like to have this feature undergo a bit more testing than the last time in order to avoid having so many people get upset at the same time. I'd really appreciate feedback for this PR so that we can catch as many problems as possible before merging.

I have really only tested in the following environment:

• Server running on WSL Ubuntu
• Client connecting via Chrome on Windows 10 (localhost:7860)

[MrKuenning]

I have been getting an error saying "error fetching model details" quite often when I select various models.

This pops up even when I have card detail view turned off.

The model still adds to the prompt, and it still works to generate images. It just throws the error each time they are selected.

In the console I get the following dump:

*** API error: GET: http://10.1.1.5:7799/sd_extra_networks/get-model-details?extra_networks_tabname=lora&model_name=Style%20Of%20Peter%20Bagge%20224 {'error': 'ZeroDivisionError', 'detail': '', 'body': '', 'errors': 'division by zero'}
    Traceback (most recent call last):
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\anyio\streams\memory.py", line 98, in receive
        return self.receive_nowait()
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\anyio\streams\memory.py", line 93, in receive_nowait
        raise WouldBlock
    anyio.WouldBlock

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\base.py", line 78, in call_next
        message = await recv_stream.receive()
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\anyio\streams\memory.py", line 118, in receive
        raise EndOfStream
    anyio.EndOfStream

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "E:\AI\A1111-Portable Dev - Copy\webui\modules\api\api.py", line 186, in exception_handling
        return await call_next(request)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\base.py", line 84, in call_next
        raise app_exc
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\base.py", line 70, in coro
        await self.app(scope, receive_or_disconnect, send_no_error)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\base.py", line 108, in __call__
        response = await self.dispatch_func(request, call_next)
      File "E:\AI\A1111-Portable Dev - Copy\webui\modules\api\api.py", line 150, in log_and_time
        res: Response = await call_next(req)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\base.py", line 84, in call_next
        raise app_exc
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\base.py", line 70, in coro
        await self.app(scope, receive_or_disconnect, send_no_error)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\cors.py", line 84, in __call__
        await self.app(scope, receive, send)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 24, in __call__
        await responder(scope, receive, send)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 44, in __call__
        await self.app(scope, receive, self.send_with_gzip)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
        raise exc
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
        await self.app(scope, receive, sender)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
        raise e
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
        await self.app(scope, receive, send)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\routing.py", line 718, in __call__
        await route.handle(scope, receive, send)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\routing.py", line 276, in handle
        await self.app(scope, receive, send)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\routing.py", line 66, in app
        response = await func(request)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\fastapi\routing.py", line 237, in app
        raw_response = await run_endpoint_function(
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\fastapi\routing.py", line 165, in run_endpoint_function
        return await run_in_threadpool(dependant.call, **values)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\starlette\concurrency.py", line 41, in run_in_threadpool
        return await anyio.to_thread.run_sync(func, *args)
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
        return await get_asynclib().run_sync_in_worker_thread(
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
        return await future
      File "E:\AI\A1111-Portable Dev - Copy\webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
        result = context.run(func, *args)
      File "E:\AI\A1111-Portable Dev - Copy\webui\modules\ui_extra_networks.py", line 1278, in get_model_details
        return JSONResponse({"html": page.gen_model_details_html(model_name)})
      File "E:\AI\A1111-Portable Dev - Copy\webui\modules\ui_extra_networks.py", line 1066, in gen_model_details_html
        model_specific = self.get_model_detail_extra_html(model_name)
      File "E:\AI\A1111-Portable Dev - Copy\webui\extensions-builtin\Lora\ui_extra_networks_lora.py", line 205, in get_model_detail_extra_html
        cmap_idx = math.floor((tag_count - min_tag) / (max_tag - min_tag) * (cmap.N - 1))
    ZeroDivisionError: division by zero

---

[Sj-Si]

@MrKuenning I think I've fixed your issues

[MrKuenning]

Seems to work now.

[freecoderwaifu]

Performance is still great and everything seems good. Kinda stopped testing because everything looked ok on my side and bugfixing seemed to be shifting to the mobile UI, which I couldn't really provide feedback on, so was just waiting for the merge.

I did find a couple of very small issues though.

Embeddings tab seems to have infinite scrolling, it goes back to the top after scrolling all the way down. Only seems to happen on that tab.

Recording.2024-06-22.124119.mp4

On the checkpoints tab, there's a weird issue that prevents scrolling all the way down. Only seems to happen when there are two full rows at the bottom, say:

XXXXX
XXXXX gets stuck

XXXXX
X doesn't get stuck

Untitled.mp4

Seems to be fixed by making the very bottom row uneven by disabling dir view, or also by resizing the Extra Networks frame from the bottom.

Untitled2.mp4

Only seems to happen on the Checkpoints tab, couldn't reproduce on the Lora tab. No visible errors on any console.

[Sj-Si]

@freecoderwaifu Go ahead and test it out again. I fixed a few bugs with scrolling. I think both your issues stemmed from the same problem so I'm hoping they're both fixed now. I don't think your issues were necessarily related to the specific tabs, I think it was just that you had just the right number of models in each of those individual tabs for the problem to show itself.

[freecoderwaifu]

@Sj-Si Thank you, it's fixed, everything else still looking good.

[Vesperindustrial]

Unsure how recent it is as I only noticed it yesterday but the tree view seems to...collapse its previous state when you go back to it from the txt2img tab (i.e. if you have folder A and A-A expanded and come back, it will all be collapsed down to just the root A). I didnt see any obvious settings that might cause it. I actually see the folders compact themselves back up when i flip back to the lora tab. No console errors appear and the actual card display view doesnt change from the previous selection, its only on the tree pane.

[Sj-Si]

@Vesperindustrial

Unsure how recent it is as I only noticed it yesterday but the tree view seems to...collapse its previous state when you go back to it from the txt2img tab (i.e. if you have folder A and A-A expanded and come back, it will all be collapsed down to just the root A). I didnt see any obvious settings that might cause it. I actually see the folders compact themselves back up when i flip back to the lora tab. No console errors appear and the actual card display view doesnt change from the previous selection, its only on the tree pane.

Fixed. I moved a variable around and missed it when updating references to it.

[Vesperindustrial]

@Vesperindustrial

Unsure how recent it is as I only noticed it yesterday but the tree view seems to...collapse its previous state when you go back to it from the txt2img tab (i.e. if you have folder A and A-A expanded and come back, it will all be collapsed down to just the root A). I didnt see any obvious settings that might cause it. I actually see the folders compact themselves back up when i flip back to the lora tab. No console errors appear and the actual card display view doesnt change from the previous selection, its only on the tree pane.

Fixed. I moved a variable around and missed it when updating references to it.

Seems to be fixed now, thanks!

[MrKuenning]

Any timeline for this being merged in?

[Sj-Si]

Any timeline for this being merged in?

no idea. that is all up to A1111 but I don't want to pressure them since this is so big and they aren't being paid to do any of this.

[w-e-w]

@Sj-Si I have some changes that I like to push
if you think it the changes are bad modify or revert them

I find it extremely annoying that the additional elements added into the generation bar / extra networks tab bar causes the tab bar height to change when switching between generation and extra networks tabs

in order to stop the changing of tab bar height I made some changes to the css
namely to the values of some width height margin padding line-height

also since for it's not possible to squeeze in the extra-network-control--header without changing the height and considering that tool tips works for PC / mouse users I don't think extra-network-control--header add much value so I remove it for PC users,
touch screen users (basically mobile uses) well still see the extra-network-control--header

befor

2024-07-24.00_55_21_028.chrome.mp4

after

2024-07-24.00_55_51_151.chrome.mp4

[w-e-w]

I also found a potential issue with this whole rework PR
I tested this on Firefox and Chrome and this issue seem to be only in Chrome

after this PR when there's lots of Extra Network items loaded

for me it only happens on my Lora tab and not Textual Inversion tab

if I switch to a different browser tab (like a different web page) when I switch back to webui, the entire web page becomes blank for for a brief second and then reappeared
I suspect some sort of too many items loaded Chrome decided to unload action is going on

while this does not impact functionality it is very annoying and distracting
and this did not happen before the PR

current dev branch without this PR, no flash / blank when swiching to webui

2024-07-24.01_34_04_765.chrome.mp4

with this PR - webpage flash / blank when swiching to webui when extra networks tab selected (Lora tab which has lots of items)

2024-07-24.01_33_00_311.chrome.mp4

[w-e-w]

ahhh my mistake I had disabled cache turned on

outdatred my mistake

another issue that impacts user experience

from what I can observe you seems to be splitting the extra networks tab into multiple chunks and load / updates the HTML chunks sequentially as the user scrolls down

while I like the concept but the implementation has some issues
name me that every time when it passes through chunk border it reloads the same assets from the server side
browser dose not cache the images so extra server calls is made
this is especially apparent when you are going back and forth on a border
as demonstrated in the video

maybe when the cache images is above a certain amount then the cash should be thrown away to clear out memory
but it should happen less often and certainly not causing image loading flashes

2024-07-24.02_29_30_723.chrome.mp4

[Sj-Si]

@w-e-w Hey I'm sorry for goin AWOL for a bit. By all means go ahead and make improvements to this PR.

I'm a bit tied up at the moment so I can't really dedicate any time to this project until I get things sorted irl first.

Also your understanding of how the model cards are loaded (loaded in chunks) is correct.

[w-e-w]

sorry for goin AWOL for a bit
everything is fine and you're totally cool, no one's getting paid no one's getting horses everything is voluntary including you and me, anyone can be "AWOL" (LOL I had to search up what that means)

---

By all means go ahead and make improvements to this PR

Thanks

---

I don't know if AUTO has reviewed this or not
I think AUTO he is also currently busy with other things recently so I'm guessing that's why he's not able to review this yet