Issues with ComfyUI crashing with and without error message using API and Flux - !!! Exception during processing !!! Given groups=1, weight of size [4, 4, 1, 1], expected input[1, 16, 90, 160] to have 4 channels, but got 16 channels instead. Flux is 16 channels, why the error! #4831
Replies: 21 comments 16 replies
-
|
There should be detailed error messages in the terminal. |
Beta Was this translation helpful? Give feedback.
-
|
No, it's like it seg faults and just drops to cmd prompt in Windows with no message (Reminds me of seg faulting in Linux). It is issues with the memory management. All these issues started after Flux support was added, Please see #4344 which has much more information. There are multiple issues now. #1 API workflows causing ComfyUI to crash when same workflow works fine in web interface. Attempting to run an exported API involving Flux causes the same seg fault type crash. #2 I used to be able to run SDXL Lightning 1920x1080 resolution through the API but now it is also causing the same seg faulting like Flux. I have a P40 and a RTX 2070. I had no issues before running SDXL in 1080p even on the RTX 2070. At some point shortly after flux was added, the memory issues were introduced. None of the released versions are working correctly regarding the memory management and the API. There was a version before the releases that had SDXL 1080p working without issue. the 1080p resolution only stopped working some point shortly after flux support was added. I have been running an image generation website for over a year and it was working fine up to about 2 months ago when I had to remove the 1080p and 2160p options as they kept crashing the interface. Would really like to get this resolved. It appears as though the API is not using the same memory management decisions and memory allocation which is causing it only to crash in the API. It always crashes at the very end of the generation where I believe it's applying the vae. The API also gave an error regarding I think the vae with the sizes not matching. My website is https://AiImageCentral.com. I also noticed in version .0.2.2 that the RemBG custom node is no longer working. I did a fresh install and for some reason it's crashing. Reverted back to 0.2.1 where it's working. |
Beta Was this translation helpful? Give feedback.
-
|
Cases where a process is killed without any error message are generally due to insufficient RAM, causing the OS to forcibly kill the process. You need to monitor the resources on task manager. |
Beta Was this translation helpful? Give feedback.
-
|
I have 72gb ram and 24gb + 8gb VRAM... SDXL worked fine before the introduction of Flux. Some condition is different in the API causing the crashes. The same workflow will work fine in the web interface but crash in the API! Without any error message. Like I said above, I believe there is a problem with the VAE when going through the API causing the crashes because I did see an error through the API saying the size didn't match. When crashing from the API, it always crashes after the last line: |
Beta Was this translation helpful? Give feedback.
-
|
It's literally a couple seconds away from completing and saving the image... |
Beta Was this translation helpful? Give feedback.
-
|
It is always crashing after the line using pytorch attention in VAE... got prompt |
Beta Was this translation helpful? Give feedback.
-
|
I have tried MANY different workflows using 3 different Flux models to try and get this working. The API is broken! Here are some of the API's I have tried to use that cause it to crash. They all work fine in the web interface but cause comfyUI to crash through the API!!! diffusion API test.json |
Beta Was this translation helpful? Give feedback.
-
|
Here is a screenshot showing SDXL working, then trying to run a Flux model and crashing with no error. I have tried on 2.3.1 and 2.4.1, I have tried with and without xformers. I have tried checkpoint,gguf, and diffusion models. I have tried on all released versions from .0.0.8 to 0.2.2.
|
Beta Was this translation helpful? Give feedback.
-
|
Here is the complete error even when using a flux 8fp model... (After successfully running SDXL) got prompt Prompt executed in 127.55 seconds |
Beta Was this translation helpful? Give feedback.
-
|
This is only one of the issues.. There are also many times it crashes without any error. |
Beta Was this translation helpful? Give feedback.
-
|
This is a typical symptom when incompatible Diffusion and VAE models are applied. |
Beta Was this translation helpful? Give feedback.
-
|
It is the correct models... I have said this like 10 times.. the workflows are fine when going through the webui (localhost in web browser)... I only get this error using the exact same workflow when going through the API. There is nothing wrong with my setup or methods or models. The same models work 100% fine through the web interface. The API is not using the same settings or configurations as the webui otherwise I would not be getting these errors. The API is where the problem is. The memory management is different.. the memory allocations are changed when going through the API |
Beta Was this translation helpful? Give feedback.
-
|
Here is a screenshot showing the command window and Flux working from the web interface. |
Beta Was this translation helpful? Give feedback.
-
|
When I go through the API, the memory allocation looks like this... loaded partially 5682.67451171875 5681.663146972656 0 and crashes The VAE I'm using is ae.safetensors for Flux like it should be along with t5xxl 8fp and 16fp versions |
Beta Was this translation helpful? Give feedback.
-
|
Here is the latest API file where that error is from.. |
Beta Was this translation helpful? Give feedback.
-
|
The Flux model I am using here is called STOIQ.4.step.8fp.safetensors (It is this model: https://civitai.com/models/161068/stoiqo-newreality-or-flux-sd-xl-lightning?modelVersionId=728048) |
Beta Was this translation helpful? Give feedback.
-
|
This clearly shows I am loading the Flux model.. You stated Flux is 16 channels and I am loading a Flux model so why am I getting an error stating 16 channels is a problem... model weight dtype torch.float8_e4m3fn, manual cast: torch.float16 |
Beta Was this translation helpful? Give feedback.
-
|
ERROR ONLY HAPPENING WHEN RUNNING FROM THE API |
Beta Was this translation helpful? Give feedback.
-
|
This is the Flux STOIQ model running from the web interface with no issues. Look at the memory allocation... This is across two runs.. same prompt.. showing first time load and second time.. got prompt |
Beta Was this translation helpful? Give feedback.
-
|
This is the API attempting to do the exact same thing crashing.. Again, look at the memory allocation.. Same Flux STOIQ model. Same workflow used just through the API. got prompt |
Beta Was this translation helpful? Give feedback.
-
|
Why the difference in memory allocation of the workflow.. [loaded completely 0.0 11340.293029785156 True] versus |
Beta Was this translation helpful? Give feedback.
-
|
Can you please look at my thread again... Loading Flux model which is 16
channels but getting an error saying 16 channels is wrong supposed to be 4
…On Sat, Sep 7, 2024 at 11:10 PM Dr.Lt.Data ***@***.***> wrote:
Cases where a process is killed without any error message are generally
due to insufficient RAM, causing the OS to forcibly kill the process.
Occasionally, it can be due to a torch issue, but this is rare.
—
Reply to this email directly, view it on GitHub
<#4831 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEVP2RC5MZML5OFTIJYQOZ3ZVO5Z7AVCNFSM6AAAAABN2KL25CVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJXHE4DCOA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
When the
|
Beta Was this translation helpful? Give feedback.
-
|
Here is the information running from my diffusion workflow... Using pytorch attention in VAE Prompt executed in 44.74 seconds |
Beta Was this translation helpful? Give feedback.
-
|
Here is the output run from the API of that exact same workflow.. Crashed as usual! No output since no generation so no logger info. Ignore the SDXL output.. I am running a website so someone just generated an SDXL image through the API on my other card. got prompt ComfyUI-Manager: installing dependencies done.** ComfyUI startup time: 2024-09-12 20:11:20.763956 Prestartup times for custom nodes: Total VRAM 24474 MB, total RAM 65464 MB |
Beta Was this translation helpful? Give feedback.
-
|
Again, something is wrong with using the API... Look at the memory usage from the workflow... loaded completely 22683.637799072265 22680.586059570312 True Why is the API saying.. loaded partially 5658.79951171875 5656.919006347656 0 |
Beta Was this translation helpful? Give feedback.
-
|
I am running the newest version of comfyui again... The new one with only one Comfy directory deep... Same error.. got prompt At least now it gets through the generation.. it's just the porblem of it it thinking 16 is wrong when it's not! Should be an easy fix. |
Beta Was this translation helpful? Give feedback.
-
|
Can you please look into fixing this.. It's been over 2 months and I'm pretty sure this is broken for everybody. I really want to get flux working through the API to add to my website.. https://AiImageCentral.com. I have tried Torch 2.3.1 2.4.0 2.4.1... |
Beta Was this translation helpful? Give feedback.
-
|
Create This will print prompt. |
Beta Was this translation helpful? Give feedback.
-
|
Done.. See results in next messages. |
Beta Was this translation helpful? Give feedback.
-
|
Here are both DBG results.. they are different even though it's exactly the same workflow I exported to API Working prompt from workflow: Debug prompt from API Even when I copy the DBG prompt from the workflow it still fails.. It's not recognizing 16 channels as valid for Flux in the API Error again capture from got prompt: got prompt Prompt executed in 84.25 seconds Running latest torch and xformers.. 2.4.1 |
Beta Was this translation helpful? Give feedback.
-
|
They are completely different prompt. first prompt uses If your API was exported from a workflow, check the timestamp of the file that was generated during the export. |
Beta Was this translation helpful? Give feedback.
-
|
I don't know what's going on, Usually when I test flux, I run from my second instance on the P40 and it runs. Also another thing messed up with the API is I can't get one of my python scripts to use port 8190 instead of 8188. When I run it, it still goes to 8188 even though I changed the port to 8190! Why does this happen and how can I fix it? do I need to have to separate installations of compyUI to run on 8190? Here is the debug.. Debug when running from port 8190.. (Works, creates Flux image) [DBG] PROMPT: {'6': {'inputs': {'text': 'a biker girl in front of a tavern, she wears a white tank top and tight leather pants, she is sensual, has a beautiful and exercised body, starry sky', 'clip': ['68', 0]}, 'class_type': 'CLIPTextEncode', '_meta': {'title': 'CLIP Text Encode (Positive Prompt)'}}, '8': {'inputs': {'samples': ['31', 0], 'vae': ['67', 0]}, 'class_type': 'VAEDecode', '_meta': {'title': 'VAE Decode'}}, '31': {'inputs': {'seed': 317409031786781, 'steps': 8, 'cfg': 1, 'sampler_name': 'euler', 'scheduler': 'beta', 'denoise': 1, 'model': ['60', 0], 'positive': ['35', 0], 'negative': ['33', 0], 'latent_image': ['64', 0]}, 'class_type': 'KSampler', '_meta': {'title': 'KSampler'}}, '33': {'inputs': {'text': '', 'clip': ['68', 0]}, 'class_type': 'CLIPTextEncode', '_meta': {'title': 'CLIP Text Encode (Negative Prompt)'}}, '35': {'inputs': {'guidance': 3.5, 'conditioning': ['6', 0]}, 'class_type': 'FluxGuidance', '_meta': {'title': 'FluxGuidance'}}, '48': {'inputs': {'filename_prefix': 'flux', 'images': ['8', 0]}, 'class_type': 'SaveImage', '_meta': {'title': 'Save Image'}}, '60': {'inputs': {'ckpt_name': 'STOIQ.4.step.8fp.safetensors', 'device': 'cuda:0'}, 'class_type': 'CheckpointLoaderMultiGPU', '_meta': {'title': 'Load Checkpoint (Multi-GPU)'}}, '64': {'inputs': {'width': 960, 'height': 544, 'batch_size': 1}, 'class_type': 'EmptyLatentImage', '_meta': {'title': 'Empty Latent Image'}}, '67': {'inputs': {'vae_name': 'ae.safetensors'}, 'class_type': 'VAELoader', '_meta': {'title': 'Load VAE'}}, '68': {'inputs': {'clip_name1': 'ViT-L-14-TEXT-detail-improved-hiT-GmP-TE-only-HF.safetensors', 'clip_name2': 't5xxl_fp16.safetensors', 'type': 'flux'}, 'class_type': 'DualCLIPLoader', '_meta': {'title': 'DualCLIPLoader'}}} Same workflow exported to API.. or even loading from workflow is crashing running 8188 now. { I'm going to try setting up another completely new instance with a dupplicate folder to run on 8190 and see if that works. |
Beta Was this translation helpful? Give feedback.
-
|
I even tried combinning all the prompt info from both toegether to see if that fixes it.. NOPE prompt_text = """ Requested to load Flux Prompt executed in 81.35 seconds |
Beta Was this translation helpful? Give feedback.
-
|
It always says given groups=1 which I think must be wrong. What should groups be equal to when 16 channels? |
Beta Was this translation helpful? Give feedback.
-
|
I also can't run SDXL in 1080p anymore either through the API as it crashes exactly the same way now! I was doing 1080p for like 8 months till this happened!!! (Like 2-2.5 months now) Now I can only do 1280x720 through the API. This should not be that difficult, it used to work! Just tried again after clean git clone again (On 9/22/2024) running 2.4.1 and everything working from workflows but API still very messed up. RTX 2070 (8gb) for SDXL generation.. P40 (24gb) for Flux if the API ever works with it and doing remBG. Both used for my website https://AiImageCentral.com. I would really appreciate if you could figure this out. I know it's not my setup. I have run through so many variations and versions... fresh installs.. git clone and released versions.. I don't think the API works with Flux for anyone. |
Beta Was this translation helpful? Give feedback.
-
|
Same issue with v0.3.7 windows portable in browser the workflow can work, be if export API json and run script with it, the server will crash without any log my workaround is get the api payload json in browser instead of use the export API json, and remove the client_id modify the code a little payload='xxx'
def queue_prompt(prompt):
data = json.dumps(payload).encode('utf-8')
req = request.Request("http://127.0.0.1:8188/prompt", data=data)
request.urlopen(req)
prompt = json.loads(prompt_text)
#set diff parameters here
prompt["prompt"]["6"]["inputs"]["text"] = "comic style"
queue_prompt(prompt) |
Beta Was this translation helpful? Give feedback.
-
I have been having issues with the API ever since Flux was added to the mix. My workflows work fine when I run them locally through the web interface but when I take the exact same API and export it as an API and then try and use it with my website, it crashes without an error message. Believe it's memory issues. This is happening when attempting to run SDXL in 1920x1080 resolution or running any of the flux models. Again they work fine through the web interface but cause ComfyUI to crash with or without error when run from the API. I have tried all releases and they are all affected. I have only been able to run 720p and 1440p resolutions without crashing for about the last 2 months now... Also in the latest release (0.2.2) now RemBG is broken as well. I have tried fresh installs like 5 times now going back to the earliest releases available and they are all broken.
Is anyone else having these issues??? I posted an issue on this like a month ago and still no reply. I am running a P40 and RTX 2070.
Beta Was this translation helpful? Give feedback.
All reactions