Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use comfyscript on modal return "ValueError: Host '127.0.0.1:8199' cannot contain ':'" #73

Closed
binarytahr opened this issue Sep 27, 2024 · 11 comments
Labels
bug Something isn't working client pending Pending ComfyUI or custom node changes

Comments

@binarytahr
Copy link

binarytahr commented Sep 27, 2024

i write my modal script according to your example script: ComfyScript/examples/modal.py at main · Chaoses-Ib/ComfyScript. modal always return error before or after comfyscript load(), may i ask for some help or advice?

my code snippets:

@app.cls(
	allow_concurrent_inputs=10,
	concurrency_limit=1,
	container_idle_timeout=30,
	timeout=300,
	gpu="T4",
)
class comfyui_flux:
	@modal.build()
	def build(self):
		print('building container...')

	@modal.enter()
	def enter(self):
		print('entering container...')
		from comfy_script.runtime import load, ComfyUIArgs
		# load()
		# load(comfyui='http://127.0.0.1:8199')
		load(args=ComfyUIArgs('--listen', '127.0.0.1', '--port', '8199'))

	@modal.exit()
	def exit(self):
		print('exiting container...')

	@modal.method()
	def inference(self, prompt, image_size, step, seed, guidance, sampler='euler', scheduler='simple', denoise=1):
		print('start inference...')
		from comfy_script.runtime import Workflow
		from comfy_script.runtime.nodes import UnetLoaderGGUF, DualCLIPLoaderGGUF, VAELoader, EmptyLatentImage, CLIPTextEncodeFlux, ConditioningZeroOut, KSampler, VAEDecode, SaveImage, LoraLoaderModelOnly
		with Workflow(queue=True, wait=True):

error in modal app logs:

Ib Custom Nodes: Loaded
ComfyScript: Loaded

Import times for custom nodes:
   0.0 seconds: /root/comfy/ComfyUI/custom_nodes/websocket_image_save.py
   0.0 seconds: /root/comfy/ComfyUI/custom_nodes/ComfyUI-GGUF
   0.3 seconds: /root/comfy/ComfyUI/custom_nodes/ComfyUI-Manager
   1.9 seconds: /root/comfy/ComfyUI/custom_nodes/ComfyScript

Starting server

To see the GUI go to: http://127.0.0.1:8199
Nodes: 225
Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_app.py", line 473, in _handle
    match_info = await self._router.resolve(request)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_urldispatcher.py", line 1010, in resolve
    match_dict, allowed = await resource.resolve(request)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_urldispatcher.py", line 755, in resolve
    not request.url.raw_path.startswith(self._prefix2)
        ^^^^^^^^^^^
  File "aiohttp/_helpers.pyx", line 26, in aiohttp._helpers.reify.__get__
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_request.py", line 437, in url
    url = URL.build(scheme=self.scheme, host=self.host)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/yarl/_url.py", line 380, in build
    netloc = cls._make_netloc(
             ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/yarl/_url.py", line 1052, in _make_netloc
    ret = cls._encode_host(host)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/yarl/_url.py", line 1033, in _encode_host
    _host_validate(host)
  File "/usr/local/lib/python3.11/site-packages/yarl/_url.py", line 1602, in _host_validate
    raise ValueError(
ValueError: Host '127.0.0.1:8199' cannot contain ':' (at position 9)
ComfyScript: Failed to watch, will retry in 5 seconds: 500, message='Invalid response status', url=URL('http://127.0.0.1:8199/ws?clientId=acb2c98c-3a8e-485d-9f72-f57dec0026bb')
Traceback (most recent call last):
  File "/root/comfy/ComfyUI/custom_nodes/ComfyScript/src/comfy_script/runtime/__init__.py", line 488, in _watch
    async with session.ws_connect(f'{client.client.base_url}ws', params={'clientId': _client_id}) as ws:
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1141, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 795, in _ws_connect
    raise WSServerHandshakeError(
aiohttp.client_exceptions.WSServerHandshakeError: 500, message='Invalid response status', url=URL('http://127.0.0.1:8199/ws?clientId=acb2c98c-3a8e-485d-9f72-f57dec0026bb')
@Chaoses-Ib
Copy link
Owner

Does load() without args also return errors?

@binarytahr
Copy link
Author

yes, the same error log

### Loading: ComfyUI-Manager (V2.51.2)
### ComfyUI Revision: 2731 [83b01f96] | Released on '2024-09-27'
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
Ib Custom Nodes: Loaded
ComfyScript: Loaded

Import times for custom nodes:
   0.0 seconds: /root/comfy/ComfyUI/custom_nodes/websocket_image_save.py
   0.0 seconds: /root/comfy/ComfyUI/custom_nodes/ComfyUI-GGUF
   0.1 seconds: /root/comfy/ComfyUI/custom_nodes/ComfyUI-Manager
   0.7 seconds: /root/comfy/ComfyUI/custom_nodes/ComfyScript
Starting server

To see the GUI go to: http://127.0.0.1:8188
Nodes: 225
Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_app.py", line 473, in _handle
    match_info = await self._router.resolve(request)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_urldispatcher.py", line 1010, in resolve
    match_dict, allowed = await resource.resolve(request)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_urldispatcher.py", line 755, in resolve
    not request.url.raw_path.startswith(self._prefix2)
        ^^^^^^^^^^^
  File "aiohttp/_helpers.pyx", line 26, in aiohttp._helpers.reify.__get__
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_request.py", line 437, in url
    url = URL.build(scheme=self.scheme, host=self.host)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/yarl/_url.py", line 380, in build
    netloc = cls._make_netloc(
             ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/yarl/_url.py", line 1052, in _make_netloc
    ret = cls._encode_host(host)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/yarl/_url.py", line 1033, in _encode_host
    _host_validate(host)
  File "/usr/local/lib/python3.11/site-packages/yarl/_url.py", line 1602, in _host_validate
    raise ValueError(
ValueError: Host '127.0.0.1:8188' cannot contain ':' (at position 9)
ComfyScript: Failed to watch, will retry in 5 seconds: 500, message='Invalid response status', url=URL('http://127.0.0.1:8188/ws?clientId=d115b379-d912-49e7-9fc3-58847b53b6bb')
Traceback (most recent call last):
  File "/root/comfy/ComfyUI/custom_nodes/ComfyScript/src/comfy_script/runtime/__init__.py", line 488, in _watch
    async with session.ws_connect(f'{client.client.base_url}ws', params={'clientId': _client_id}) as ws:
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1141, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 795, in _ws_connect
    raise WSServerHandshakeError(
aiohttp.client_exceptions.WSServerHandshakeError: 500, message='Invalid response status', url=URL('http://127.0.0.1:8188/ws?clientId=d115b379-d912-49e7-9fc3-58847b53b6bb')
start inference...
result <comfy_script.runtime.data.NodeOutput object at 0x7ecadd029e50> <comfy_script.runtime.factory.Image object at 0x7ecadd029d90>
Error handling request
Traceback (most recent call last):

@binarytahr
Copy link
Author

大佬你每次回复好像都好快

@Chaoses-Ib
Copy link
Owner

Chaoses-Ib commented Sep 27, 2024

yarl 12 小时前的更新引入了破坏性更改(aio-libs/yarl#954 ),导致 aiohttp 在 URL 含端口时会出错(aio-libs/aiohttp#9307 ),可以先用 yarl 旧版(pip install yarl==1.12.1)或者等 aiohttp 新版发布。ComfyUI 本身也用了 aiohttp,不知道受不受影响。

@Chaoses-Ib Chaoses-Ib added pending Pending ComfyUI or custom node changes client bug Something isn't working labels Sep 27, 2024
@binarytahr
Copy link
Author

非常感谢~我周末试一下

@binarytahr
Copy link
Author

aiohttp 3.10.7 似乎解决这个问题了,明天我试一下

@binarytahr
Copy link
Author

binarytahr commented Sep 29, 2024

验证了,pip install aiohttp==3.10.8 解决问题。感谢帮助~

@binarytahr
Copy link
Author

现在可以正常运行推理了,但是最后返回的结果会报错。请问是我应该本地也要安装 PIL 或者 comfyui & comfyscript 吗?我现在本地什么都没安装。

class comfyui_flux_server:
	@modal.build()
	def build(self):
		print('building container...')

	@modal.enter()
	def enter(self):
		print('entering container...')
		from comfy_script.runtime import load, ComfyUIArgs
		load(args=ComfyUIArgs('--listen','127.0.0.1','--port','8199'))

	@modal.exit()
	def exit(self):
		print('exiting container...')

	@modal.method()
	def inference(self, prompt, image_size, step, seed, guidance, sampler='euler', scheduler='simple', denoise=1):
		print('start inference...')
		from comfy_script.runtime import Workflow
		from comfy_script.runtime.nodes import UnetLoaderGGUF, DualCLIPLoaderGGUF, VAELoader, EmptyLatentImage, CLIPTextEncode, CLIPTextEncodeFlux, ConditioningZeroOut, KSampler, VAEDecode, SaveImage, LoraLoaderModelOnly
		with Workflow(queue=True, wait=True):
			model = UnetLoaderGGUF(unet_name='flux.1_dev_city96_q8_0.gguf')
			clip = DualCLIPLoaderGGUF(clip_name1='clip-vit-large-patch14_flux.1_comfyanonymous.safetensors', clip_name2='t5xxl_flux.1_city96_v1.1_q8_0.gguf', type='flux')
			vae = VAELoader(vae_name='vae_flux.1_black-forest-labs.safetensors')
			latent = EmptyLatentImage(width=image_size[0], height=image_size[1], batch_size=1)
			conditioning_positive = CLIPTextEncodeFlux(clip_l=prompt, t5xxl=prompt, guidance=guidance, clip=clip)
			conditioning_negative = CLIPTextEncode(text='', clip=clip)
			latent = KSampler(model=model, seed=seed, steps=step, cfg=1, sampler_name=sampler, scheduler=scheduler, positive=conditioning_positive, negative=conditioning_negative, latent_image=latent, denoise=denoise)
			image = VAEDecode(samples=latent, vae=vae)
			result = SaveImage(image, 'comfyui')
		print(type(image))
		print(type(result))
		return image

如果我让它 return image,modal服务端是正常的,我的本地会报错 DeserializationError: Deserialization failed because the 'comfy_script' module is not available in the local environment.

---------------------------------------------------------------------------
DeserializationError                      Traceback (most recent call last)
Cell In[12], line 3
      1 modal.Cls.lookup(app_name="comfyui_flux_server", tag="comfyui_flux_server")().inference.remote(prompt='a dog', image_size=[512,512], step=10, seed=0, guidance=3)

File ~/Downloads/modal/.pixi/envs/default/lib/python3.12/site-packages/synchronicity/synchronizer.py:537, in Synchronizer._wrap_proxy_method.<locals>.proxy_method(self, *args, **kwargs)
    535 instance = self.__dict__[synchronizer_self._original_attr]
    536 try:
--> 537     return wrapped_method(instance, *args, **kwargs)
    538 except UserCodeException as uc_exc:
    539     raise uc_exc.exc from None

File ~/Downloads/modal/.pixi/envs/default/lib/python3.12/site-packages/synchronicity/combined_types.py:28, in FunctionWithAio.__call__(self, *args, **kwargs)
     26     return self._func(*args, **kwargs)
     27 except UserCodeException as uc_exc:
---> 28     raise uc_exc.exc from None

File ~/Downloads/modal/.pixi/envs/default/lib/python3.12/site-packages/modal/object.py:242, in live_method.<locals>.wrapped(self, *args, **kwargs)
    239 @wraps(method)
    240 async def wrapped(self, *args, **kwargs):
    241     await self.resolve()
--> 242     return await method(self, *args, **kwargs)

File ~/Downloads/modal/.pixi/envs/default/lib/python3.12/site-packages/modal/functions.py:1321, in _Function.remote(self, *args, **kwargs)
   1316 if self._is_generator:
   1317     raise InvalidError(
   1318         "A generator function cannot be called with `.remote(...)`. Use `.remote_gen(...)` instead."
   1319     )
-> 1321 return await self._call_function(args, kwargs)

File ~/Downloads/modal/.pixi/envs/default/lib/python3.12/site-packages/modal/functions.py:1266, in _Function._call_function(self, args, kwargs)
   1258 invocation = await _Invocation.create(
   1259     self,
   1260     args,
   (...)
   1263     function_call_invocation_type=api_pb2.FUNCTION_CALL_INVOCATION_TYPE_SYNC_LEGACY,
   1264 )
   1265 try:
-> 1266     return await invocation.run_function()
   1267 except asyncio.CancelledError:
   1268     # this can happen if the user terminates a program, triggering a cancellation cascade
   1269     if not self._mute_cancellation:

File ~/Downloads/modal/.pixi/envs/default/lib/python3.12/site-packages/modal/functions.py:187, in _Invocation.run_function(self)
    183 item: api_pb2.FunctionGetOutputsItem = (
    184     await self.pop_function_call_outputs(timeout=None, clear_on_success=True)
    185 ).outputs[0]
    186 assert not item.result.gen_status
--> 187 return await _process_result(item.result, item.data_format, self.stub, self.client)

File ~/Downloads/modal/.pixi/envs/default/lib/python3.12/site-packages/modal/_utils/function_utils.py:467, in _process_result(result, data_format, stub, client)
    464     raise RemoteError(result.exception)
    466 try:
--> 467     return deserialize_data_format(data, data_format, client)
    468 except ModuleNotFoundError as deser_exc:
    469     raise ExecutionError(
    470         "Could not deserialize result due to error:\n"
    471         f"{deser_exc}\n"
    472         "This can happen if your local environment does not have a module that was used to construct the result. \n"
    473     )

File ~/Downloads/modal/.pixi/envs/default/lib/python3.12/site-packages/modal/_serialization.py:358, in deserialize_data_format(s, data_format, client)
    356 def deserialize_data_format(s: bytes, data_format: int, client) -> Any:
    357     if data_format == api_pb2.DATA_FORMAT_PICKLE:
--> 358         return deserialize(s, client)
    359     elif data_format == api_pb2.DATA_FORMAT_ASGI:
    360         return _deserialize_asgi(api_pb2.Asgi.FromString(s))

File ~/Downloads/modal/.pixi/envs/default/lib/python3.12/site-packages/modal/_serialization.py:116, in deserialize(s, client)
    111         raise DeserializationError(
    112             f"Deserialization failed with an AttributeError, {exc}. This is probably because"
    113             " you have different versions of a library in your local and remote environments."
    114         ) from exc
    115 except ModuleNotFoundError as exc:
--> 116     raise DeserializationError(
    117         f"Deserialization failed because the '{exc.name}' module is not available in the {env} environment."
    118     ) from exc
    119 except Exception as exc:
    120     if env == "remote":
    121         # We currently don't always package the full traceback from errors in the remote entrypoint logic.
    122         # So try to include as much information as we can in the main error message.

DeserializationError: Deserialization failed because the 'comfy_script' module is not available in the local environment.

如果我让它 return result,modal会报错 TypeError: cannot pickle '_contextvars.Context' object

Prompt executed in 65.11 seconds
<PIL.PngImagePlugin.PngImageFile image mode=RGB size=512x512 at 0x7EF6A677A390>
Queue remaining: 1
Queue remaining: 0
<class 'comfy_script.runtime.factory.Image'>
<class 'comfy_script.runtime.data.NodeOutput'>
<PIL.PngImagePlugin.PngImageFile image mode=RGB size=512x512 at 0x7EF695633E10>
Queue remaining: 0
Traceback (most recent call last):
  File "/pkg/modal/_container_io_manager.py", line 772, in handle_input_exception
    yield
  File "/pkg/modal/_container_entrypoint.py", line 425, in run_input_sync
    container_io_manager.push_outputs(
  File "/pkg/synchronicity/synchronizer.py", line 537, in proxy_method
    return wrapped_method(instance, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/pkg/synchronicity/combined_types.py", line 28, in __call__
    raise uc_exc.exc from None
  File "/pkg/modal/_container_io_manager.py", line 852, in push_outputs
    *[self.format_blob_data(self.serialize_data_format(d, data_format)) for d in data]
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/pkg/modal/_container_io_manager.py", line 852, in <listcomp>
    *[self.format_blob_data(self.serialize_data_format(d, data_format)) for d in data]
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/pkg/modal/_container_io_manager.py", line 506, in serialize_data_format
    return serialize_data_format(obj, data_format)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/pkg/modal/_serialization.py", line 346, in serialize_data_format
    return serialize(obj)
           ^^^^^^^^^^^^^^
  File "/pkg/modal/_serialization.py", line 89, in serialize
    Pickler(buf).dump(obj)
  File "/pkg/modal/_vendor/cloudpickle.py", line 1227, in dump
    return super().dump(obj)
           ^^^^^^^^^^^^^^^^^
TypeError: cannot pickle '_contextvars.Context' object
exiting container...

@binarytahr
Copy link
Author

ComfyScript/examples/modal.py at main · Chaoses-Ib/ComfyScript
示例里的 print("result", result) 在我这里没返回结果

@Chaoses-Ib
Copy link
Owner

result 是个任务,要先 wait 等它完成,再 wait 来获取图片 list:

from PIL import Image

images: list[Image.Image] = result.wait().wait()

这部分文档在 https://github.com/Chaoses-Ib/ComfyScript/blob/main/docs/Images/README.md#saving 。这个写法确实有点绕,后面可能会把惰性获取图片去掉来简化成 wait 一次,应该也没多少场景用得到。

Modal 支持不支持序列化 PIL.Image.Image 我不确定,如果不支持的话还要自己序列化下,可以用 _repr_png_() 转成 bytes。

@binarytahr
Copy link
Author

binarytahr commented Sep 29, 2024

谢谢说明!我明天试一下。我暂时用的笨方法,读取本地图片转成 bytes。我之前都是用的real模式,这个模式不了解,没注意到还要 wait。

for file_ in pathlib.Path('/root/comfy/ComfyUI/output').iterdir():
	if file_.name.startswith(file_name_prefix):
		return file_.read_bytes()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working client pending Pending ComfyUI or custom node changes
Projects
None yet
Development

No branches or pull requests

2 participants