Libthai and new steamwebhelper

[garpu]

Your system information

• Steam client version (build number or date): March 20, 2019 18:38:53, package versions 1443121671
• Distribution (e.g. Ubuntu): Slackware 14.2, multilib
• Opted into Steam client beta?: Yes
• Have you checked for system updates?: Yes

Please describe your issue in as much detail as possible:

I updated the client to today's new build, and the store won't load with the following error in console:
./steamwebhelper: error while loading shared libraries: libthai.so.0: cannot open shared object file: No such file or directory

Library loads, and I can launch games. I can't open my friend list, store, communities, or profile. I don't use the Thai language. Is there a way to disable it? Or shouldn't this come with the Steam runtime libs?

Steps for reproducing this issue:

1. Start Steam
2. Load the store.

[garpu]

Installing libthai fixed the issue. I seem to recall there being an option for the Thai language before in Steam. Did a library get left out of the runtime?

[ghost]

Had the very same problem on Gentoo. Was able to get around it by making custom ebuilds for libdatrie and libthai, since non of them exists in the Gentoo repository or any overlays that I could find.
ebuilds for those who may need them:
libdatrie
libthai

Your system information

• Steam client version (build number or date): March 20, 2019 18:38:53, package version 1553121671
• Distribution (e.g. Ubuntu): Gentoo, multilib
• Opted into Steam client beta?: Yes
• Have you checked for system updates?: Yes

[sylware]

custom light gnu/linux distro without libthai, exactly the same issue

[0xC0ncord]

On Gentoo GNU/Linux with the same issue. @Fureloka's ebuilds fixed the problem instantly as soon as they were merged; didn't even need to restart Steam.

Your system information

• Steam client version (build number or date): March 20, 2019 18:38:53, package version 1553121671
• Distribution (e.g. Ubuntu): Gentoo amd64 multilib
• Opted into Steam client beta?: Yes
• Have you checked for system updates?: Yes

[chewi]

I'll consider adding the ebuilds to the official Gentoo tree at the weekend but Pango automagically links against libthai so we need to fix that first.

[kisak-valve]

Hello, per "Fix regression causing black pages for web based components for some Linux users due to missing dependencies in new Chromium update" in the 2019-03-21 Steam client beta update, please retest this issue.

Note: Running steam without the steam runtime will still need this dependency and #6156 handled by the distro's package maintainer(s) for Steam.

[sylware]

Got the update of the beta client: libthai still missing even with the new steam runtime, and indeed, "anything webcomponent" is dark.

[garpu]

For the sake of testing, I uninstalled libthai, and I'm still getting the error that libthai is missing. This is with the build pushed today (2019-03-21).

I've got STEAM_RUNTIME_PREFER_HOST_LIBRARIES set to false, so I'm definitely using the steam runtime.

ETA: reinstalling libthai fixes it.

[natethor]

On Solus 4.0 I'm also having this same issue. I tested with the Steam Client and the Beta Update, reinstalled multiple times. I also tried running "STEAM_RUNTIME_PREFER_HOST_LIBRARIES=0 steam" and "STEAM_RUNTIME=1 steam", neither worked as a work-around.

OS: Solus 4.0 fortitude
Kernel: x86_64 Linux 4.9.163-127.lts

[sylware]

wait for the update which will include libthai and its deps, and everything should work again.

[garpu]

I thought this update was supposed to have the updates?

[sylware]

give them time

[TheDaftRick]

Newest update and still have this issue on Solus

Built: Mar 21 2019, at 21:47:17
Steam API: v018
Steam package versions: 1553209175

Taking libdatrie.so.1 and libthai.so.0 from my Ubuntu machine and placing it in the /Steam/ubuntu12_64/ folder fixes the issue

[sylware]

once those deps are packaged with the steam client web component (I guess blink, the google fork of webkit for chrome), everything will work again on non-thai enabled distros. !!! Just thought about it: the licences of those deps: MIT or BSD or Lesser GPL? Coze if it's plain GPL (without the lesser exception), valve is not allowed to distribute them since they link with closed source programs. Wild guess without checking: it's lesser GPL and distribution is ok.

[kisak-valve]

Hello, per "Additional missing web component dependencies." in the 2019-03-22 Steam client beta update, please retest this issue.

[garpu]

I uninstalled libthai and libdatie, and it works again for me. No problems with steamwebhelper.

[sylware]

still broken, because libharfbuzz.so.0 is now missing: ./steamwebhelper: error while loading shared libraries: libharfbuzz.so.0: cannot open shared object file: No such file or directory

[DerRidda]

Also fixed for me in the latest update.

[vmisupov]

Have this issue on Gentoo too.
Steam and Dota starts, but Dota cant start match and Steam Friend Chat does not work.

./ubuntu12_32/steam-runtime.old/i386/usr/lib/i386-linux-gnu/pango/1.6.0/modules/pango-thai-lang.so
	libthai.so.0 => not found
./ubuntu12_32/steam-runtime.old/amd64/usr/lib/x86_64-linux-gnu/pango/1.6.0/modules/pango-thai-lang.so
	libthai.so.0 => not found
./ubuntu12_32/steam-runtime/i386/usr/lib/i386-linux-gnu/pango/1.6.0/modules/pango-thai-lang.so'
	libthai.so.0 => not found

Have topic on forum:
https://forums.gentoo.org/viewtopic-p-8320322.html

[vmisupov]

I'll consider adding the ebuild

Have you reported to bugs.gentoo? Need add ebuilds to portage and add deps for steam in wiki

[sylware]

This is not specific to gentoo. Steam webcomponent and/or steam runtime is missing some dependencies: was libthai, now it's libharfbuzz.

[chewi]

No need to report to Gentoo, I'm already here. 😛 @anyc dealt with libthai and libdatrie in steam-overlay but as mentioned, they've now been bundled anyway. Having just switched to the beta myself, I'm now missing libffi.so.6 and libselinux.so.1. I also see that libpcre.so.3 can be an issue.

To be honest, all this broken bundling is intensely frustrating. I'm already having to deal with the fact that Gentoo doesn't want to package ancient libgcrypt 1.5 any more, which is needed for old Source Engine games. I am the author of esteam, which helps to reduce some of these issues but it is currently only applied to games, not Steam itself. We've tried to cover the libraries Steam requires with regular dependencies but that's looking increasingly difficult. I'll see whether I can address this in the launcher script.

[sylware]

You mean that there is more than libharbuzz missing from the steam client/runtime?

[chewi]

Yes, although most people will have libharfbuzz anyway, it's a hard requirement of GTK3.

Options are limited. Changing or removing files in this directory results in Steam automatically repairing them before it even starts. I thought about LD_LIBRARY_PATH but this directory is always prepended to it. The only solution I've found is LD_PRELOAD. It's not great but it works. I'll try and put something together.

[sylware]

Yes, although most people will have libharfbuzz anyway, it's a hard requirement of GTK3.

I don't have GTK3, and "most people will have libharfbuzz.." is a very wrong reason, because if it was a good one, we would not have a gnu/linux port in the first place. So, I can expect more missings deps once libharfbuzz is packaged in the steam client/runtime, pepe hands.

[vmisupov]

No need to report to Gentoo, I'm already here. stuck_out_tongue @anyc dealt with libthai and libdatrie in steam-overlay but as mentioned, they've now been bundled anyway. Having just switched to the beta myself, I'm now missing libffi.so.6 and libselinux.so.1. I also see that libpcre.so.3 can be an issue.

To be honest, all this broken bundling is intensely frustrating. I'm already having to deal with the fact that Gentoo doesn't want to package ancient libgcrypt 1.5 any more, which is needed for old Source Engine games. I am the author of esteam, which helps to reduce some of these issues but it is currently only applied to games, not Steam itself. We've tried to cover the libraries Steam requires with regular dependencies but that's looking increasingly difficult. I'll see whether I can address this in the launcher script.

Thank you very much for your work!)

[TTimo]

In the interest of reducing the deployment and testing churn on this, here are two more libraries that we will deploy in the next beta Steam Client update:

83f9537995bf3231d94bdc06adb56ffb  libharfbuzz.so.0
65477ec91c7650351891103fc2453ad5  libgraphite2.so.3

https://www.dropbox.com/s/bq6f7w3byz9aaf3/webhelper-runtime-20190325.tgz?dl=0

We hope that this is the last set of libraries that we include.

Libraries such as libselinux.so.1 and libpcre.so.3 that I see mentioned are already included in the steamrt:scout runtime. We believe that if you are seeing errors with those, it is because you are using a distribution-altered steam package, and we will not chase those further.

Several of the distribution maintainers are active on here, so please work this out with them, preferrably on the distribution's own bug trackers.

We anticipate to bring significant changes to the runtime setup over the next few months, I would encourage everyone to setup the Steam Client with as little distribution customization as possible.

[chewi]

@TTimo, knowing who you are, I hold a good deal of respect and I know that what I say carries little weight here but I feel the need to say this anyhow.

As a distro developer, I would much prefer that you bundled less, not more. This bundling largely seems to be to support libcef.so. Few, if any, distros seem to package it so it is reasonable that you would bundle it. Gentoo's excuse is that Chromium itself is hard work to maintain and takes forever to build so I don't really want to burden myself and the users with libcef as well. The libraries it directly needs are a very commonly available set though and none are any of the missing libraries mentioned above. These missing libraries are further optional subdependencies that would be avoided if you simply used the system copies of glib and Pango. Now I can imagine that you may have built Pango with libthai because you undoubtedly have Thai users to support. Gentoo, on the other hand, has seemingly got this far down the road without any Thai users complaining and even if we did support libthai, we wouldn't force it on everybody. So while I can accept that you may also need to bundle the likes of libthai, Steam's automatic repair feature also makes it very awkward for us to run it without. Please bear this in mind.

As for libpcre.so.3, you can hardly blame us for that as this soname is a Debian invention that is not recognised by upstream or any non-Debian distros. I had to make a compatibility package with a symlink.

[chewi]

I've just gone back to an older system to check what it looked like before and it was as I suggested above. libcef.so bundled with the system left to deal with glib and Pango, among others. I'm curious to know why this changed.

[sylware]

On Mon, Mar 25, 2019 at 11:20:14AM -0700, TTimo wrote: We anticipate to bring significant changes to the runtime setup over the next few months, I would encourage everyone to setup the Steam Client with as little distribution customization as possible.

Users of minimal gaming gnu/linux distros can be very usefull benchmarks. (I'll gladly help fine tune the distribution packages) I can already give some advices. core gamer system dependencies on linux based OSes: 1 - elf dynamic loader/DNS network configuration: provided by the "libc". user level public customization of elf dynamic loader configuration is provided by LD_* environment variables. But there is a private non customizable part (don't count on /etc/ld.conf for instance, and expect a "system" content of ld_* variables). On steamos, a network configuration interface would be expected, which is the case with a networkmanager stack. big caveat: there is no binary compatibility between gnu libc and other libc, for instance musl libc. This is very sad, this is a failure of core ABI. Then gnu libc ABI (since elf ABI is not enough), would be the way to go. Don't expect the gnu libc "programs" being available to a gamer user level account. If valve deploys its own DNS servers, it could provide it's own network stack on top of the linux kernel network ABI. But I don't think it's a good idea (DNS servers DDOS). 2 - On a gcc toolchain: compile everything with --static-libgcc and --static-libstdc++. The gamer system libgcc_s will be loaded by the gamer system glibc for pthread_cancel to work. g++ and libgcc_s ABIs are HELL, stay as far as you can from them. I did solve tons of 3D drivers issues since my custom distro is compiled with those flags... but nearly all the gnu/linux distros out there do not, so it's more reasonable to put those compilation flags on valve and games side. 3 - do not expect to run any system administration level programs (could be hidden at the user level). 4 - sound, the alsa lib (pulseaudio is not to be expected since libasound could use dmix or pulseaudio for mixing). The gamer system alsa lib should be loaded, since its deployment and configuration could be specific and hardcoded into the libs. For instance, there is no way to tell a third party libasound to fetch it's data files elsewhere than the path, usually /usr/share/alsa, hardcoded at compilation time. There is still a possible conflict between applications using the gamer system alsa lib with those distributed by valve and using the steam runtime alsa lib, for instance if dmix is configured differently. 5 - input methods and windowing system. WSI: x11 or wayland (android is either wayland or native). In a perfect world, it should be dynamically discovered or forced by the gamer (xwayland...). The gamer system input methods should be used since it could be heavily customized (keyboard layout, virtual keyboard, CJK input methods...). Some configuration paths are hardcoded into x11 libs at build time, then if distributed, configuration file system location mistmatches could happen (for instance /usr/share/x11 is hardcoded in some x11 client libs). 6 - 3D APIs. GL or vulkan. In a perfect world, it should be dynamically discovered with a 100% software fallback based on the WSI type (some gamers may run the client on some systems unable to run their video games, for administrative purpose or social purpose). 7 - expect only a basic shell (for instance dash or busybox ash), and a mimimal set of user level posix commands. Be very conservative in their usage. Don't hesitate to code and distribute some programs to help your shell scripts (actually the right way to do it). Don't expect any high level script engines on the gamer system: python2/python3, perl5/perl6, lua, node.js (javascript), squirel, php, ruby, haskell, etc (PLZ!). But valve could distribute its prefered ones, ofc. 8 - do not expect a specific GFX toolkit to be there: GTK+, Qt, EFL, TCL_TK... only WSI native should be expected. 9 - do not expect a huge www browser like firefox or chromium (don't expect anything javascript). Basically, it should work with curl... 10 - do not count on the Linux Standard Base: only huge mainstream distros with an army of devs and integrators can achieve this. This is really not reasonable for small/medium distros. 11 - expect clean 64bits distros without 32bits support. 12 - everything else should be packaged and distributed by valve/games. A good way to work on a binary distribution package, would be to do that in a chroot jail with system dependencies carefull accounted for, build and configured in "non-standard" file system location (file system fuzzing). You would start with: - only the glibc elf interpreter in /lib (the only real absolute ABI requirement) - LD_* and PATH environment variables with very few commands. - WSI environment variables (DISPLAY for x11, I don't recall the one for wayland). strace tool will help a lot there. Do _NOT_ use the file system hierarchy of a huge/mainstream gnu/linux distro or you will miss tons of things and may work _only_ on this specific huge/mainstrem distro.

[TTimo]

It is unfortunate that my comment about distribution alteration of the Steam packages being harmful is taking center stage here. Let's focus on the issue at hand instead.

Since no one has chimmed in on harfbuz and graphite we'll go ahead and assume that they are the last two libraries that we needed to include. This is confirmed by testing, the remaining libraries pulled from the host are core libraries and/or the graphics stack which wouldn't appropriate (or possible) to bundle.

Finally, a bit of background information regarding the runtime environment that the steamwebhelper process is expected to run in:

The LD_LIBRARY_PATH is setup so that ubuntu12_64/ is searched first, then the steamrt:scout runtime is searched (ubuntu12_32/steam-runtime/amd64/) and finally the host.

By comparison with the production client, the only difference is the addition of the first ubuntu12_64/ lookup. The reason is that the new CEF can no longer be easily compiled against the steamrt:scout runtime, but it hasn't changed so much that it requires a complete new runtime, so this is an adequate intermediate solution.

[sylware]

I have not get any steam beta client update. I'm still stuck with a missing libharfbuzz.

[TTimo]

I have not get any steam beta client update. I'm still stuck with a missing libharfbuzz.

That's because I posted the libraries ahead of time to this thread, so we can test it out without going through a full beta client update cycle, in case more iteration is needed. I think we will update the beta client today though.

[sylware]

Jez, could not test: dropox wants aka javascri pt. Do you have a direct link? ... I wait the update and pray that all the deps are distributed :( (libharfbuzz could depend on a finit-state automaton lib, rygel, if I recall properly, but it was a long time ago, maybe this dep is gone).

[sylware]

Maybe you could drop the file into a public github gist? (I always forget about gist) Did you check harfbuzz dep on rygel finite-automaton lib?

[cpages]

In the interest of reducing the deployment and testing churn on this, here are two more libraries that we will deploy in the next beta Steam Client update:

83f9537995bf3231d94bdc06adb56ffb  libharfbuzz.so.0
65477ec91c7650351891103fc2453ad5  libgraphite2.so.3

https://www.dropbox.com/s/bq6f7w3byz9aaf3/webhelper-runtime-20190325.tgz?dl=0

We hope that this is the last set of libraries that we include.

@TTimo I confirm that those were the last missing deps for me (NixOS). Thanks!

[chewi]

It is unfortunate that my comment about distribution alteration of the Steam packages being harmful is taking center stage here. Let's focus on the issue at hand instead.

Sorry, harfbuzz and graphite2 are libraries that Gentoo packages so we were already covered there.

Finally, a bit of background information regarding the runtime environment that the steamwebhelper process is expected to run in:

The LD_LIBRARY_PATH is setup so that ubuntu12_64/ is searched first, then the steamrt:scout runtime is searched (ubuntu12_32/steam-runtime/amd64/) and finally the host.

By comparison with the production client, the only difference is the addition of the first ubuntu12_64/ lookup. The reason is that the new CEF can no longer be easily compiled against the steamrt:scout runtime, but it hasn't changed so much that it requires a complete new runtime, so this is an adequate intermediate solution.

Thanks for the explanation, that makes sense. Steam used to have a variable called STEAM_RUNTIME_PREFER_HOST_LIBRARIES but I gather that's no longer used because host libraries are preferred by default. This only applies to games though. Could we have an equivalent for the platform libraries?

[sylware]

Got the update. Fixed here too.

[TTimo]

Steam used to have a variable called STEAM_RUNTIME_PREFER_HOST_LIBRARIES but I gather that's no longer used because host libraries are preferred by default.

It still works. That's a debug feature meant to be set to 0 to rely the runtime as much as possible and disable library pinning. On mesa-based graphics stacks that will fail because you lose working drivers, but it can be useful for developers on nvidia systems.

Thanks for testing!

[chewi]

Steam used to have a variable called STEAM_RUNTIME_PREFER_HOST_LIBRARIES but I gather that's no longer used because host libraries are preferred by default.

It still works. That's a debug feature meant to be set to 0 to rely the runtime as much as possible and disable library pinning.

Sorry, yes, I was tired and forgot the specifics. My point was that it would be useful to have a similar variable like STEAM_PLATFORM_PREFER_HOST_LIBRARIES that defaults to 0 but could be set to 1.