A new release for soundfile #325
Comments
|
Thanks for taking time to solve this, If u need any helping hands let me know |
|
I'm not sure about the specific CLI flags. I think it's adding The platform tags would be something like For my own projects I have to rely on GitHub Actions to build MacOS packages/libraries for me as downloadable artifacts. I can write a workflow for you if you're unable to compile the arm64 library yourself. |
|
Thank you so much for your help! If you could point me to an example of how to have Github compile the macOS libraries, that would be fantastic! That might actually allow us to compile the wheels separately on each OS, instead of the current franken-system that packages precompiled libraries with the generic python code. |
|
This would be an example of a workflow. This script would be stored in a path like name: Compile libraries
on:
push:
pull_request:
jobs:
build-libs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: ["macos-11"]
fail-fast: true
steps:
- uses: actions/checkout@v2
- name: Compile library
run: ./mac_build.sh
- uses: actions/upload-artifact@v2
with:
name: libs
path: "*.dylib"
retention-days: 7
if-no-files-found: error I imagine this running the GitHub has good documentation on making workflows. Build artifacts uploaded this way can be downloaded from another job, such as a job that builds the wheel. |
|
This worked amazingly well, thank you a whole lot! At the moment, Github runners do not yet provide M1 machines, but that's not super important at the moment, since @chuma9615 and @jurihock have already gracefully provided pre-build libraries in libsndfile-binaries#7 and libsndfile-binaries#8, respectively. I'll investigate cross-compilation once I've managed to build wheels. |
|
I have drafted a new release at releases/0.11.0b1. There is a known issue that will require a recompile of the Windows binaries with Visual Studio 14+, and will probably never work for all versions of Python. Apparently, file descriptor open does only work if libsndfile is compiled against the same Microsoft C Runtime as Python. The present libsndfile-provided binaries are apparently incompatible with my Python. If anyone would like to contribute up-to-date Python libraries compiled with Visual Studio 14+, I'd be grateful for a pull request in bastibe/libsndfile-libraries. Or better yet, integrate it into the CI scripts to build the binaries automatically. Also, you would help me tremendously if anyone could test the provided wheels on macOS and M1! |
|
Will you provide source distribution archives (tarballs) for the latest and upcoming releases or should Linux distro packagers use the source archives automatically created by Github? |
|
I will upload a source distribution. The reason I didn't attach it to the draft release linked above is that the current build script still bundles all the libraries with the source dist, making it unnecessarily big. This will need fixing before the final release is cut. |
Looks like if you compile to v140 then that binary will support all versions of Python after 3.5. The instructions I've found for telling MinGW to use specific runtimes seems convoluted so maybe it would be easier to use Vcpkg to get these builds. |
I'm inexperienced with all the required compilation steps and CI. But I remembered that Mathias uses a script which builds a small version of libsndfile on Mac during some Github actions, build-small-libsndfile.sh. Forgive me if this doesn't help at all. :) |
|
@SpotlightKid I just uploaded a fixed sdist to the latest tag. Right now, all that's missing for a release is
@HaHeho thank you for your response. Thanks to @HexDecimal, we now have automated builds for Windows and macOS! That's wild! |
If you check the commits on GitHub then TravisCI's last status check was Nov 26, 2019. It stopped working sometime after that, probably when they dropped support for the |
Thanks for the heads up. The URLs for the attached files still have |
|
@bastibe Tested the wheel on a M1 machine and it's working Note that I tested soundfile-0.11.0b2-py2.py3-none-any.whl
|
|
@chuma9615 that's the source-only wheel. Could you try the M1-specific soundfile-0.11.0b2-py2.py3-none-macosx-10.x-arm64.whl? |
Note that The libs were built with How wheel tags work is clear, but MacOS specific info is harder to find: |
|
@bastibe When using the macOS ARM wheel it crashes |
|
I was trying to package python-soundfile 1.11.0 (https://github.com/bastibe/python-soundfile/tree/bastibe/0.11.0) for Nix on macOS arm64, and encountered the Full logAfter replace According to https://cffi.readthedocs.io/en/latest/using.html#callbacks, |
|
Thank you all so much for your help with this release! And especially thank you @HexDecimal, for implementing the CI builds and tests! I've uploaded a new set of pre-release wheels: 0.11.0b3. I hope that these wheels now work on macOS M1. Again, I'd be grateful if anyone with access to an M1 Mac could test the macOS ARM wheel. |
MacBook Pro (13-inch, M1, 2020), macOS 12.2.1 |
I've looked into this, and actually drafted an implementation of the callbacks locally. However, there is a problem with this approach: Currently, we are using the ABI mode of CFFI. The "new style" callbacks you mentioned require that soundfile uses the API mode of CFFI, which means compiling a bespoke version of soundfile for every operating system and version of Python we support. While I am certainly open to doing this, especially now that we have CI runners to automate the task, I think it is out-of-scope for this pull request. Please raise the issue in a new issue, or preferably, try to draft a pull request for it. |
I've seen a similar error on my wife's old mac. But it seemed to be related to her old version of pip. Perhaps that was wrong. Does your error go away if you install a current version of pip? Regardless, the issue still needs fixing. I set the OS tag to Could you confirm the correct OS tag to use? Perferably with some measure of backwards and forwards compatibility. Anyway, my time to work on this for this week is up, so I'll have to defer the issue for next wednesday. |
No.
|
|
I changed the platform tags to
Please check them out again in the 0.11.b3 beta release. If I remember correctly, there was no M1 before macOS 11.0, so we don't need to include the 10.9 tag, right? |
It works after upgrading pip: |
|
Thank you for the confirmation! Does anyone know why we need an up-to-date pip for this? |
How tags are handed can change from version to version. My guess is that older versions of Pip are looking for exact MacOS versions rather than compatible versions or something similar. If that's the case then adding redundant versions would solve the issue for older versions of pip. The links I've posted previously should help: #325 (comment), mostly with which MacOS versions Python might be expecting, which could be I think adding tags for |
|
It seems that the macOS x64 wheel somehow links to some system-installed /usr/local/opt/lame/lib/libmp3lame.0.dylib, instead of our own libmp3lame.la. I've tried a fix in libsndfile-binaries#14, but apparently that didn't work as intended and still links against /usr/local/opt/lame. Does anyone understand what's going on here? As far as I'm aware that's the last blocker for the release. |
|
I can help test on either Darwin arch if that's useful |
|
Thank you for the offer! The build issue on macOS/Intel is still blocking, and I haven't had time to work on it. Currently, libsndfile-binaries@11 links against /usr/local/opt/lame instead of the self-compiled liblame from build_mac.sh. If anyone wants to try to fix this issue in the build script, or contribute a self-contained libsndfile with full MP3-support for macOS/Intel, we could get this ready for release. |
|
This issue may be related and one to watch: |
|
Thanks to the outstanding work of @joetoddsonos in libsndfile-binaries@15, we should now have working macOS/Intel binaries with statically-linked MP3 support. As far as I know, this was the last blocking problem for the 0.11 release 🤞. Once again, I ask for your help, and try out the new wheels of python-soundfile-0.11.0b6. If these finally work on all supported platforms, we can publish the next release of soundfile with MP3 support! Thank you all so much for your help and patience! This release wouldn't have been possible without your invaluable help! |
|
I just uploaded the wheels to Pypi, finally! We should have working MP3 support in soundfile, now! |
It's still failing for me :( Python 3.10.4 [conda env.] Case 1 Output Case 2 Output CC: @bastibe |
|
How did you install soundfile? You will need both soundfile 0.11.0, and a version of libsndfile that supports MP3. The wheels from Pypi do contain such a libsndfile, but your system's libraries or conda's might not. |
soundfile version: 0.11.0 I tried 3 ways to install:-
It's working for python 3.85 :). I can settle with this. |
|
Please manually download soundfile-0.11.0-py2.py3-none-macosx_10_9_arm64.macosx_11_0_arm64.whl from https://pypi.org/project/soundfile/#files. Then install the wheel using Otherwise, check if soundfile is picking up your system's libsndfile instead of the wheel-provided one. Library load order is a bit of a fickle beast sometimes. |
|
Thanks @bastibe & co for this update! I was looking at opening a PR to update the conda-forge recipe, and for that, it would be most helpful if there was also a source dist (tar.gz) on pypi. Would it be possible to include that in a post-release? |
|
Dear @bmcfee, thank you for alerting me to that. I simply forgot to upload the source dist. It's available now. |
|
Excellent, thank you! |
I've tried everything else and still can't load mp3 files. I've downloaded a version of libsndfile that supports mp3 but how do I check if soundfile is picking up the right libsndfile version and not a system one? Was using soundfile as the audio backend for torchaudio, but even when I use soundfile.read it gives the same error |
|
System error almost always means your file name is incorrect. It seems you are missing the backslash after |
Omg thanks, didn't see that. |
|
Try opening the file with straight soundfile instead of going through torch. Who knows what torch is doing in the background. |
|
I'm facing the same issue: Here is my code: |
|
@asennoussi Ubuntu 22.04 still packages libsndfile version 1.0.31, which, AFAIK, does not have MP3 support yet. Also note that, due to errors in the CMake build files, the pen-ultimate release of libsndfile (1.1.0) does not have MP3 support in most Linux distribution packages either, even though this version does support MP3 in theory, but due to said errors it will not be compiled in. The latest version (1.2.0) fixes that, but this still has to arrive in Ubuntu. |
|
Makes sense? Instead. I hope this helps out someone else. |
|
I have compiled a preliminary binary wheel for Ubuntu recently, that I'll try to release for testing soon. This should allow MP3 support on Ubuntu if installed with pip, and if no system-libsndfile is present. |
|
Please check out #364 for a beta-release of a binary wheel for Linux. |
|
@bastibe hello, I am so fraisated keep got this error: soundfile.py", line 1226, in _init_virtual_io I installed from pypi, tried your M1 arm wheel, all fails. Why? I also tried unsintall previous just keep error |
|
I am still getting the same error even with version 0.12.1. Is there a solution to this issue? |
and others
This is a continuation of the discussion in #310, where I'm working on packaging a new version of soundfile.
It appears that the packaging problem is more complicated than anticipated. Our existing build system will probably need to be replaced, which will take some time.
If anyone would like to help me package soundfile, I'd be grateful!
The problem is, we need to package four wheels for
each containing their own libsndfile{.dll,.dylib}, and making sure that the right version gets installed on the right platform.
We have a build script in place for building packages for 1-3, but I don't yet know how to build wheels specifically for macOS Arm, and it seems that we should convert our build system to use PEP 517 (pyproject.toml/build et al).
I only had an hour or so today to work on this, so there wasn't much progress. I'll keep you updated.
The text was updated successfully, but these errors were encountered: