Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could glib be avoided? #41

Open
skuzzymiglet opened this issue Jun 26, 2020 · 15 comments
Open

Could glib be avoided? #41

skuzzymiglet opened this issue Jun 26, 2020 · 15 comments
Labels
build Build framework dist Distribution (packaging, binary builds, etc) feature New feature or request

Comments

@skuzzymiglet
Copy link

I'm concerned that glib might not be available on every platform. Could it be removed as a dependency? (I'm not a C dev, I'm just wondering)

@hpjansson
Copy link
Owner

It could, but one of the reasons I'm using it is to improve portability in the first place (for instance, it lets Chafa work with both pthreads and Windows threads). One possible solution would be to copy a small part of GLib into Chafa, but that would increase the maintenance burden and code size, and I'd have to take special care to make sure the exported API doesn't clash with applications that also use GLib proper.

Which platforms are you concerned about in particular?

@skuzzymiglet
Copy link
Author

skuzzymiglet commented Jun 26, 2020 via email

@hpjansson
Copy link
Owner

It's relative, I guess. GLib is well-maintained, well-documented, widely used (e.g. GNOME stack) and has had regular releases for over 20 years. In addition to threading, I also need it for Unicode tables, option parsing, data structures, error reporting, precondition and versioning macros, general string support, etc. The C stdlibs don't reliably provide all of those things, so every project is forced to either reinvent the wheel (poorly) or use one or more libraries like GLib.

That said, Chafa only uses the core GLib stuff (no GIO or GObject), so it'd be possible to drop in a cut-down GLib similarly to what pkg-config does: https://gitlab.freedesktop.org/pkg-config/pkg-config/ . You'd then be able to configure Chafa with --with-internal-glib.

I'm curious to know how much interest there is in this. If it's something people commonly want to do, I can give it priority.

@hpjansson
Copy link
Owner

@skuzzymiglet Just for fun, I set up a static build yesterday. This pulls the code from GLib, ImageMagick, libjpeg, libpng and a few others into the chafa binary itself so it can just be copied into any Linux image as needed.

Static builds will appear here: https://hpjansson.org/chafa/releases/static/ (via https://hpjansson.org/chafa/download/).

I used LTO to eliminate unreferenced symbols, but even so, the resulting binary is a serious chonk. Here's the symbol table from gcc-nm -S --size-sort -r chafa (size in 2nd column): chafa-1.4.1-1-x86_64-linux-gnu.sym.txt

Most of this comes from ImageMagick, which has tons of image processing code we don't need referenced from function tables. More proof the ImageMagick dep needs to go away (and it will, with time). GLib, on the other hand, hardly registers.

I realize this is somewhat tangential to what you're asking (as I understand it, you want to reduce build-time deps), but this was easy to do and can hopefully be used as a workaround.

@oakes
Copy link

oakes commented Sep 18, 2021

@hpjansson is there a repo or makefile somewhere that you use to create these static builds? I want to try making a static build myself rather than using the precompiled one you have, just to experiment with.

@hpjansson
Copy link
Owner

@oakes Well... It exists (barely), but it's not in good shape for sharing. I adapted a least-effort shell script I made for another project years ago, which I run in a VM instance that's set up just right, making it unlikely to work anywhere else.

If you're still interested, feel free to write me an e-mail and I can help you get set up, but you should know that it'll require some time investment on your end too :)

If I were to do it properly (and had the time), I'd probably switch to Docker or LXC.

@oakes
Copy link

oakes commented Oct 8, 2021

I managed to get my own static build working, but my ultimate goal is to target emscripten so i can use this awesome library in a browser. But even when building statically i'm hitting a wall because glib just isn't really emscripten-friendly. Lots of static asserts checking for 64 bit support, while emscripten / wasm is currently 32 bit. And also i think it's using some headers that aren't part of musl like linux/futex.h.

I think glib was a decent choice for the platforms you had in mind but since i'm keen on targeting the browser i am thinking i should try assessing what it would take to write a shim that replaces it with something else, maybe a collection of smaller libs that are easier to build with emscripten. If you have any pointers let me know :D

@hpjansson
Copy link
Owner

I'll help. What's the quickest way I can get set up with emscripten and reproduce the build errors? I'm in touch with the GLib developers and have contributed a bit there -- they're very responsive and good collaborators, and part of GLib's job is to be a portability layer, so it might be possible to fix the issues upstream.

Failing that we could look at a bundled build. pkg-config has one that might be a good starting point: https://gitlab.freedesktop.org/pkg-config/pkg-config

The bundled glib would have to be optional with a configure switch, since most distro packages will want to use the platform glib (avoiding issues that might arise from applications mixing Chafa's bundled glib with the platform one).

@oakes
Copy link

oakes commented Oct 9, 2021

You can find my fork here: https://github.com/oakes/chafa

There is a build_macos.sh and build_linux.sh which should produce executables with glib statically linked. To try building with emscripten, you'll need to do this:

git clone https://github.com/emscripten-core/emsdk
cd emsdk
./emsdk install latest
./emsdk activate latest
# add the dirs that are printed by the last command to your PATH
cd ../chafa
CC=emcc ./build_linux.sh

On my system this produces...

In file included from tests/example.c:1:
In file included from chafa/chafa.h:24:
In file included from glib/glib/glib.h:30:
In file included from glib/glib/galloca.h:32:
glib/glib/gtypes.h:463:3: error: static_assert failed due to requirement 'sizeof(unsigned long long) == sizeof(unsigned long)' "Expression evaluates to false"
  G_STATIC_ASSERT(sizeof (unsigned long long) == sizeof (guint64));
  ^               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
glib/glib/gmacros.h:819:31: note: expanded from macro 'G_STATIC_ASSERT'
#define G_STATIC_ASSERT(expr) _Static_assert (expr, "Expression evaluates to false")
                              ^               ~~~~
1 error generated.

@hpjansson
Copy link
Owner

Thanks. Looks like the platform config files are incorrect. I managed to generate almost-correct config files by applying the attached glib-configure-with-emscripten.patch to a pristine GLib on Linux. Edit emscripten.txt for your environment, then run:

meson setup build --cross-file emscripten.txt -Diconv=libc

It should generate config.h etc. before failing. I had to manually edit gnulib_math.h to remove the overrides for isinf().

I'm also attaching a patch set chafa-oakes-emscripten.patch with the resulting fixes to your repo. Apply, then:

CC=emcc ./build_linux.sh
node --experimental-wasm-threads chafalinux

It's a bit hacky, but it should run and the ouput looks ok here.

glib-configure-with-emscripten.patch.txt
chafa-oakes-emscripten.patch.txt

(Sorry for the .patch.txt files, but GitHub won't let me fork your repo while already having a similarly-named repo, and it doesn't like the .patch extension in attachments either).

@oakes
Copy link

oakes commented Oct 10, 2021

That was a huge help, thank you. I wasn't able to run it in firefox but it looks like that's due to its lack of wasm thread support. I might try removing the thread use entirely to allow running it there. edit: I guess it supports them when the right headers are sent down but still will probably look into removing threads so it works in more browsers.

@hpjansson
Copy link
Owner

hpjansson commented Oct 10, 2021

Making threads optional at run time would be simple, I just haven't gotten around to it yet. It would have the form of some kind of API call e.g. chafa_set_n_threads_max(int n_threads). If set to <= 1, it would circumvent thread pools entirely and just use the main thread.

We could also add a build setting to not compile in thread support at all, but I guess that's not necessary as long as the run time call is there?

@hpjansson
Copy link
Owner

I added chafa_get_n_threads() and chafa_set_n_threads() to the master branch. If you call chafa_set_n_threads(1) at the start of your program, it should be doing everything in the main thread. Looks like you still need to run it with node --experimental-wasm-threads, though... Maybe we need that build option after all to avoid linking with pthreads, or maybe glib is initializing threads on startup.

@hpjansson
Copy link
Owner

The attached patch, applied on top of 335eb49, seemed to do the trick.

0001-Build-without-threads.patch.txt

@oakes
Copy link

oakes commented Oct 11, 2021

Perfect -- with that patch it works in chrome & FF. Thank you again for the help.

@hpjansson hpjansson added feature New feature or request build Build framework dist Distribution (packaging, binary builds, etc) labels Jul 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Build framework dist Distribution (packaging, binary builds, etc) feature New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants