Failed to build on macOS (m2 max, sonoma 14.2) with stablediffusion enabled.

[expnn]

LocalAI version:

commit: 7641f92

Environment, CPU architecture, OS, and Version:

Mac Studio 2023
M2 max 32 GB
mac0S sonoma 14.2

$ uname -a
Darwin _<hidden info>_ 23.2.0 Darwin Kernel Version 23.2.0: Wed Nov 15 21:55:06 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6020 arm64

Describe the bug

It cannot be built with stablediffusion enabled on my macOS box. There are serveral issues resulting to the failure. I fixed part of them.

1. Header issues:

   • Missing opencv headers. I solved this issue by installing the opencv library by homebrew:

     brew install opencv

     and create a symbolic link to it in the sources/go-stable-diffusion/ directory manually.

   • The ncnn header file paths are incorrect in the sources/go-stable-diffusion/stablediffusion.hpp file, which will result in header file not found error.

2. Library issues:
   I cannot link to the gomp library. I modified the
   sources/go-stable-diffusion/stablediffusion.go to make it to link to the omp library, after installing it with

   brew install libomp

3. Link issue:
   It reports "ld: multiple SYMDEF member files found in an archive in '/path/to/LocalAI/sources/go-stable-diffusion/libstablediffusion.a'", which is beyond my ability to solve the problem.

To Reproduce

1. Follow this page to install dependencies for macOS.
2. Install extra dependencies not covered by the abolve page.

   brew install opencv libomp

3. Build

   make BUILD_TYPE=metal GO_TAGS="tts stablediffusion" build

   (It will fail)
4. Apply the patch attached and create symbolic links to libraries.

   cd sources/go-stable-diffusion/
   git apply modifications.patch
   ln -s /opt/homebrew/include/opencv4/opencv2
   ln -s /opt/homebrew/Cellar/opencv/4.8.1_5/lib/libopencv_core.a
   ln -s /opt/homebrew/Cellar/opencv/4.8.1_5/lib/libopencv_imgcodecs.a
   ln -s /opt/homebrew/Cellar/libomp/17.0.6/lib/libomp.a

5. Build again. It will report the "multiple SYMDEF member files" issue.

Expected behavior

Successful build with 'stablediffusion' enabled.

Logs

Additional context

modifications.patch

[localai-bot]

⚠️⚠️⚠️⚠️⚠️

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

but.... I can also be funny or helpful 😸 and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me,
I don't support (yet) replying!

⚠️⚠️⚠️⚠️⚠️

Thank you for your report! We will review it and get back to you as soon as possible.

Sources:

• https://github.com/go-skynet/LocalAI/tree/master/CONTRIBUTING.md
• https://github.com/go-skynet/LocalAI/tree/master/.github/ISSUE_TEMPLATE/bug_report.md

[Cyb3rDudu]

I have the same issue. Any idea how to solve it?

[barracuda156]

Any update on this? I got it on Sonoma with another software, but linker error looks identical.

[Cyb3rDudu]

No, I was not able to compile it with stable diffusion enabled. Meanwhile I could not even get the latest version compiled anymore. But I haven't had time to look into it in the last 4 weeks, maybe things changed again.

[Cyb3rDudu]

No, I was not able to compile it with stable diffusion enabled. Meanwhile I could not even get the latest version compiled anymore. But I haven't had time to look into it in the last 4 weeks, maybe things changed again.

I reproduced the issue. Still

# github.com/go-skynet/LocalAI/backend/go/image/stablediffusion
/opt/local/lib/go/pkg/tool/darwin_arm64/link: running clang++ failed: exit status 1
ld: warning: ignoring duplicate libraries: '-lc++', '-lcblas'
ld: multiple SYMDEF member files found in an archive in '/Users/builder/Repos/LocalAI/sources/go-stable-diffusion/libstablediffusion.a'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[Cyb3rDudu]

Aye, I could build it with stable diffusion support.

after I had the failure, I checked the archive with

ar -tv libstablediffusion.a
rw-r--r--     501/20       266168 Feb  1 16:01 2024 __.SYMDEF
rw-r--r--     501/20       279528 Feb  1 14:51 2024 stablediffusion-hires.o
rw-r--r--     501/20       279528 Feb  1 14:51 2024 stablediffusion.o
rw-r--r--     501/20       210600 Feb  1 14:51 2024 __.SYMDEF
rw-r--r--     501/20         3200 Feb  1 14:51 2024 absval.cpp.o
rw-r--r--     501/20         4656 Feb  1 14:51 2024 absval_arm.cpp.o

as we can see, 2 SYMDEF Files, which is what the error message says.

In the makefile for go-stable-diffusion we create a list of files from the folder unpack/*

libstablediffusion.a: stablediffusion.o unpack $(EXTRA_TARGETS)
	ar src libstablediffusion.a stablediffusion-hires.o stablediffusion.o $(shell ls unpack/* | xargs echo)

In this folder, there exists a file called __.SYMDEF which I removed.
After that, I could compile and use stable diffusion on apple silicon

curl http://localhost:8080/models/apply -H "Content-Type: application/json" -d '{
  "url": "github:go-skynet/model-gallery/stablediffusion.yaml"
}'
{"uuid":"16536132-c0e3-11ee-b160-22eff34a93be","status":"http://localhost:8080/models/jobs/16536132-c0e3-11ee-b160-22eff34a93be"}

curl http://localhost:8080/v1/images/generations -H "Content-Type: application/json" -d '{
  "prompt": "A cute baby sea otter",
  "size": "512x512"
}'
{"created":1706779767,"id":"a67c7fdc-f49a-49cd-8091-4fc02161104b","data":[{"embedding":null,"index":0,"url":"http://localhost:8080/generated-images/b644172613977.png"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

Seems like the linker on linux just takes the first SYMDEF file and tolerates multiple of them in the archive while on Mac, the linker fails and does not link against an archive with multiples of them..

[holnburger]

Currently it's still failing, but with another error message:

43 warnings and 1 error generated.
make[2]: *** [stablediffusion.o] Error 1
make[1]: *** [sources/go-stable-diffusion/libstablediffusion.a] Error 2
# github.com/mudler/go-stable-diffusion
In file included from stablediffusion.cpp:14:
sources/go-stable-diffusion/stable-diffusion/x86/vs2019_opencv-mobile_ncnn-dll_demo/vs2019_opencv-mobile_ncnn-dll_demo/diffusion_slover.cpp:283:35: warning: format specifies type 'int' but the argument has type 'size_type' (aka 'unsigned long') [-Wformat]
sources/go-stable-diffusion/stable-diffusion/x86/vs2019_opencv-mobile_ncnn-dll_demo/vs2019_opencv-mobile_ncnn-dll_demo/diffusion_slover.cpp:420:33: warning: format specifies type 'int' but the argument has type 'size_type' (aka 'unsigned long') [-Wformat]
In file included from stablediffusion.cpp:23:
sources/go-stable-diffusion/stablediffusion.hpp:12:10: fatal error: 'ncnn/benchmark.h' file not found
make: *** [backend-assets/grpc/stablediffusion] Error 1

Even with your fixes applied @jamu85 (it looks like some of the fixes have been reverted from main) - thank you for your work!

[AmauryD]

just cloned https://github.com/Tencent/ncnn in sources/go-stable-diffusion removed the error. But i still have some linking errors on my side.