Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to install pgloader on macOS (Big Sur, M1) #1312

Open
Wilson13 opened this issue Aug 25, 2021 · 37 comments
Open

Unable to install pgloader on macOS (Big Sur, M1) #1312

Wilson13 opened this issue Aug 25, 2021 · 37 comments

Comments

@Wilson13
Copy link

Hi,

I was trying to install pgloader via this command:
brew install --HEAD pgloader

This is the output and error reported:

==> Cloning https://github.com/dimitri/pgloader.git
Updating /Users/wilson/Library/Caches/Homebrew/pgloader--git
==> Checking out branch master
Already on 'master'
Your branch is up to date with 'origin/master'.
HEAD is now at 3047c9a Typo fix in MS SQL documentation (#1242)
==> Patching
Error: Failure while executing; `patch -g 0 -f -p1` exited with 1. Here's the output:
can't find file to patch at input line 3
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|--- a/local-projects/cl-csv/parser.lisp
|+++ b/local-projects/cl-csv/parser.lisp
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored

Environment

MacBook Pro (13-inch, M1, 2020)
macOS Big Sur v11.5.2 (20G95)

@Wilson13 Wilson13 changed the title Unable to install pgloader on MacOS (Big Sur, M1) Unable to install pgloader on macOS (Big Sur, M1) Aug 25, 2021
@scrobot
Copy link

scrobot commented Sep 1, 2021

The same problem. I've caught exception when I tried to install pgloader on my mac(M1) with brew

brew install pgloader
Error: pgloader: no bottle available!
You can try to install from source with:
  brew install --build-from-source pgloader
Please note building from source is unsupported. You will encounter build
failures with some formulae. If you experience any issues please create pull
requests instead of asking for help on Homebrew's GitHub, Twitter or any other
official channels.

And also when I tried to build it from sources

brew install --build-from-source pgloader
==> Downloading https://ghcr.io/v2/homebrew/core/sbcl/manifests/2.1.8
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/sbcl/blobs/sha256:df642cb657479a6ae84a6adfc22cafbcb876542fb9b69527b4ca2755ffc15e25
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:df642cb657479a6ae84a6adfc22cafbcb876542fb9b69527b4ca2755ffc15e25?se=2021-09-01T11%3A05%3A00Z&sig=PRsr08H9%2BDUlBx%2FU63PTpQiPMQgbju3I
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/buildapp/manifests/1.5.6_3
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/buildapp/blobs/sha256:3a4d0ac54096f853b8b271e02369dd2666e5b69c975d3b3f18ae6e54b4adf966
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:3a4d0ac54096f853b8b271e02369dd2666e5b69c975d3b3f18ae6e54b4adf966?se=2021-09-01T11%3A05%3A00Z&sig=9VlHNiuNx%2Bbb5%2BRioAZYiIjflXOfsV5O
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/m4/manifests/1.4.19
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/m4/blobs/sha256:ea1be04e51645f9e31c8d2cab6d144bb7f47efb1f35214b9f1014e27db5a3bc1
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:ea1be04e51645f9e31c8d2cab6d144bb7f47efb1f35214b9f1014e27db5a3bc1?se=2021-09-01T11%3A05%3A00Z&sig=hVtRGKxtmCfglpaOvqRC1XKyMvvGHqMgGFPI
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/libtool/manifests/2.4.6_4
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/libtool/blobs/sha256:a41a4872cdfaa34bb4723e728b73dd8c7a05725501a262bb41ad9af4e2fcd1d6
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:a41a4872cdfaa34bb4723e728b73dd8c7a05725501a262bb41ad9af4e2fcd1d6?se=2021-09-01T11%3A05%3A00Z&sig=WaStvjRIFMgQGjKclvKcX169YFmx8IHSjoXv
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/unixodbc/manifests/2.3.9_1
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/unixodbc/blobs/sha256:66e4b186a19526e02782557afe6926d2cfb9f372e94cbcc387f531b122f510e0
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:66e4b186a19526e02782557afe6926d2cfb9f372e94cbcc387f531b122f510e0?se=2021-09-01T11%3A05%3A00Z&sig=8gpUHK35tO%2Ba0RMi88ZmjufjVxLHHJIpfS
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/freetds/manifests/1.3.3
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/freetds/blobs/sha256:95f5ec9318ef76fc7fd19e1e4bfbb651ce0f70c0a445734ea2eb52c9431b5300
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:95f5ec9318ef76fc7fd19e1e4bfbb651ce0f70c0a445734ea2eb52c9431b5300?se=2021-09-01T11%3A05%3A00Z&sig=cfKiL1UHLyRHRYB7UKJtRwIqVOYKN9A%2FdW
######################################################################## 100.0%
==> Downloading https://github.com/dimitri/pgloader/releases/download/v3.6.2/pgloader-bundle-3.6.2.tgz
==> Downloading from https://github-releases.githubusercontent.com/594087/a86b1500-6c8d-11ea-86c4-5dd3e9eae544?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210901%2Fus-east-1%2Fs3%2Faws4_r
######################################################################## 100.0%
==> Installing dependencies for pgloader: sbcl, buildapp, m4, libtool, unixodbc and freetds
==> Installing pgloader dependency: sbcl
==> Pouring sbcl--2.1.8.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/sbcl/2.1.8: 1,325 files, 60MB
==> Installing pgloader dependency: buildapp
==> Pouring buildapp--1.5.6_3.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/buildapp/1.5.6_3: 4 files, 39.9MB
==> Installing pgloader dependency: m4
==> Pouring m4--1.4.19.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/m4/1.4.19: 13 files, 726.3KB
==> Installing pgloader dependency: libtool
==> Pouring libtool--2.4.6_4.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/libtool/2.4.6_4: 75 files, 3.7MB
==> Installing pgloader dependency: unixodbc
==> Pouring unixodbc--2.3.9_1.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/unixodbc/2.3.9_1: 46 files, 2.3MB
==> Installing pgloader dependency: freetds
==> Pouring freetds--1.3.3.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/freetds/1.3.3: 1,262 files, 14.2MB
==> Installing pgloader
==> Patching
==> make
Last 15 lines from /Users/aleksejskrobot/Library/Logs/Homebrew/pgloader/01.make:
                         --load local-projects/pgloader-3.6.2/src/hooks.lisp              \
                         --load-system pgloader                    \
                         --eval '(setf pgloader.params::*version-string* "3.6.2")' \
                         --entry pgloader:main                        \
                         --dynamic-space-size 4096              \
                         --compress-core                         \
                         --output bin/pgloader.tmp
;; loading system "cffi"
;; loading system "cl+ssl"
WARNING: /opt/homebrew/Cellar/sbcl/2.1.8/libexec/bin/sbcl is loading libcrypto in an unsafe way
fatal error encountered in SBCL pid 13689 pthread 0x1002d7d40:
SIGABRT received.

   0:     0x16fd43d18 pc=0x18cccce60
make: *** [bin/pgloader] Error 1

Do not report this issue to Homebrew/brew or Homebrew/core!

@made-indrayana
Copy link

made-indrayana commented Sep 15, 2021

I have the same problem with @Wilson13, I'm on Bigs Sur 11.6 but on Intel i7.

@jhult
Copy link

jhult commented Sep 22, 2021

Same issue:

  • macOS: 12.0 Beta (21A5506j)
  • Intel i7

@csadai
Copy link

csadai commented Sep 29, 2021

Same problem with Big Sur (11.5) and Apple M1 chip

@erika-e
Copy link

erika-e commented Nov 24, 2021

I don't get the exact same error. I have Rosetta2 enabled on all of my terminal windows and installed brew from a Rosetta2 terminal.

I can run brew install pgloader but trying to run any pgloader commands e.g. pgloader --help results in the libcrytpo error message:

WARNING: /usr/local/bin/pgloader is loading libcrypto in an unsafe way
fatal error encountered in SBCL pid 53056 pthread 0x1006fb600:
SIGABRT received.

Anyone know of a workaround? I am on an M1 chip and Monterey.

@chrise86
Copy link

chrise86 commented Dec 8, 2021

I don't get the exact same error. I have Rosetta2 enabled on all of my terminal windows and installed brew from a Rosetta2 terminal.

I can run brew install pgloader but trying to run any pgloader commands e.g. pgloader --help results in the libcrytpo error message:

WARNING: /usr/local/bin/pgloader is loading libcrypto in an unsafe way
fatal error encountered in SBCL pid 53056 pthread 0x1006fb600:
SIGABRT received.

Anyone know of a workaround? I am on an M1 chip and Monterey.

Also getting this on a regular Intel based Mac mini.

@ScottSmudger
Copy link

Same issue

brew install pgloader
brew install --HEAD pgloader
and building from source doesnt work

Monterey 12.0.1 / Intel i7

Any workarounds or fixes?

@zmitja
Copy link

zmitja commented Dec 14, 2021

This fixes the issue:

brew install openssl
sudo ln -s /opt/homebrew/Cellar/openssl@3/3.0.0_1/lib/libcrypto.dylib /usr/local/lib/libcrypto.dylib
sudo ln -s /opt/homebrew/Cellar/openssl@3/3.0.0_1/lib/libssl.dylib /usr/local/lib/libssl.dylib

Found a solution here:
cl-plus-ssl/cl-plus-ssl#114

Note, I already had pgloader installed and working, upgrade to Monterey broke it, so I had the issue with running pgloader, not installing it, but I think it's the same issue so this fix should fix the installation problem as well.

@tuanalumi
Copy link

tuanalumi commented Dec 15, 2021

I am on an Intel, and I have to use Docker version to overcome this.

@yveshield
Copy link

brew install openssl@3
brew link --force openssl@3

then its works again

@nbsoftware
Copy link

nbsoftware commented Dec 23, 2021

Still not working here.

Had first to update the links to the current version of openssl:

sudo ln -s /opt/homebrew/Cellar/openssl@3/3.0.1/lib/libcrypto.dylib /usr/local/lib/libcrypto.dylib
sudo ln -s /opt/homebrew/Cellar/openssl@3/3.0.1/lib/libssl.dylib /usr/local/lib/libssl.dylib

as well as

sudo ln -s /opt/homebrew/lib/libsybdb.dylib /usr/lib/libsybdb.dylib

but then it hung shortly after:

==> Patching
==> make
Last 15 lines from /Users/fmeyer/Library/Logs/Homebrew/pgloader/01.make:
;; loading system "cffi"
;; loading system "cl+ssl"
;; loading system "mssql"
;; loading file #P"/private/tmp/pgloader-20211223-23208-t6qbpi/pgloader-bundle-3.6.2/local-projects/pgloader-3.6.2/src/hooks.lisp"
;; loading system "pgloader"
Fatal TYPE-ERROR:
  The values
    (#<FUNCTION (LAMBDA (ESRAP::PRODUCTION)
                  :IN
                  "/private/tmp/pgloader-20211223-23208-t6qbpi/pgloader-bundle-3.6.2/.brew_home/.cache/common-lisp/sbcl-2.1.9-macosx-arm64/private/tmp/pgloader-20211223-23208-t6qbpi/pgloader-bundle-3.6.2/software/cl-abnf-20190521-git/abnf.fasl") {7008B7C14B}>
     NIL NIL)

  are not of type
    (VALUES FUNCTION &OPTIONAL)
make: *** [bin/pgloader] Error 1

This is where I'm stuck.
Any ideas?

Meanwhile I'll give Ubuntu arm64 a try...

@yveshield
Copy link

@nbsoftware

I'm wondering if you completely uninstalled pgloader and openssl@3 before reinstalling?
If not, please try

@nbsoftware
Copy link

well I did try to remove the links, brew remove both, brew clean, and reinstalled, still same error :/

@nyudenkov
Copy link

Same issue here.

MacOS 12.0.1 / M1

@codr36
Copy link

codr36 commented Jan 19, 2022

Same error on Monterey 12.1 (21C52) M1 Air...
FYI installing and linking openssl@3 did not work 😕

@sawirricardo
Copy link

Any workaround? Also on monterey. No bottle available!

@nbsoftware
Copy link

My workaround was to install an ARM Ubuntu VM and get the package running there... not exactly the most trivial setup, but at least it worked

@dbouman
Copy link

dbouman commented Feb 8, 2022

I'm on macOS Monterey 12.2 M1 and getting exact same issue, installing and linking openssl did not work.

@Shantarli
Copy link

Same here with Monterey 12.3 on Intel.

@surjithctly
Copy link

Same issue with Mac Monterey with M1 Chip. What's the best solution so far?

@ifree-zm
Copy link

I have the same problem with @scrobot ,how to fix it?

@smilenannan
Copy link

might be the easiest way around.

@ScottSmudger
Copy link

ScottSmudger commented Feb 21, 2022

I circumvented it for now by using the docker image, saves the headache

@liujchen
Copy link

liujchen commented Mar 2, 2022

might be the easiest way around.

Yes, it works

@mohammed7aafar
Copy link

You can try to install from source with:

$ brew install --build-from-source pgloader

  • Please note building from source is unsupported. You will encounter build
    failures with some formulae. If you experience any issues please create pull
    requests instead of asking for help on Homebrew's GitHub, Twitter or any other
    official channels.

@nbsoftware
Copy link

➜  ~ brew install --build-from-source pgloader
==> Downloading https://github.com/dimitri/pgloader/releases/download/v3.6.2/pgloader-bundle-3.6.2.tgz
Already downloaded: /Users/fmeyer/Library/Caches/Homebrew/downloads/2a1b2485b3a52a8aa90117254bcc5b1701cd81cc53b0e6f1c16900b1d967f948--pgloader-bundle-3.6.2.tgz
==> Patching
==> make
Last 15 lines from /Users/fmeyer/Library/Logs/Homebrew/pgloader/01.make:
;; loading system "cffi"
;; loading system "cl+ssl"
;; loading system "mssql"
;; loading file #P"/private/tmp/pgloader-20220309-61828-1p1i4fg/pgloader-bundle-3.6.2/local-projects/pgloader-3.6.2/src/hooks.lisp"
;; loading system "pgloader"
Fatal TYPE-ERROR:
  The values
    (#<FUNCTION (LAMBDA (ESRAP::PRODUCTION)
                  :IN
                  "/private/tmp/pgloader-20220309-61828-1p1i4fg/pgloader-bundle-3.6.2/.brew_home/.cache/common-lisp/sbcl-2.1.9-macosx-arm64/private/tmp/pgloader-20220309-61828-1p1i4fg/pgloader-bundle-3.6.2/software/cl-abnf-20190521-git/abnf.fasl") {7008DBC14B}>
     NIL NIL)

  are not of type
    (VALUES FUNCTION &OPTIONAL)
make: *** [bin/pgloader] Error 1

@rgossiaux
Copy link

might be the easiest way around.

Just going to bump this comment because this actually works, if you are running into this.

@plus-
Copy link

plus- commented Apr 1, 2022

I know it's not ideal but for people using docker I've built the image on my M1 and it's available here

@szynkowskij
Copy link

szynkowskij commented Apr 1, 2022

Deleted my comment, but the make did end up working just fine once I found out my work laptop has an enforced proxy server on it 🙃

Building 3.6.3 is definitely the way right now. I think I used brew to install sbcl, clisp openssl, freetds and I'm not sure I even needed those, but part of the problem I think I suffered a lot from was having an x86_64 version of brew running through rosetta as well and I think just incorrect x86 libs were being picked up instead of arm64.

Hopefully we'll see a brew build get updated. I think the main issue with it is that it's still on 3.6.2 and not utilizing newer versions of pgloader lisp dependencies from quicklisp.

@adimyth
Copy link

adimyth commented Apr 14, 2022

Same issue

brew install pgloader brew install --HEAD pgloader and building from source doesnt work

Monterey 12.0.1 / Intel i7

Any workarounds or fixes?

This doesn't seem to be working for me. I'm on M1 Pro & using version 3.6.3

The make pgloader command is stuck forever & doesn't make any progress

@szynkowskij
Copy link

Same issue
brew install pgloader brew install --HEAD pgloader and building from source doesnt work
Monterey 12.0.1 / Intel i7
Any workarounds or fixes?

This doesn't seem to be working for me. I'm on M1 Pro & using version 3.6.3

The make pgloader command is stuck forever & doesn't make any progress

It making and getting stuck with no feedback seems odd if you just skim through the makefile. Do you have some of the additional requirements? Like sbcl?

@adimyth
Copy link

adimyth commented Apr 28, 2022

Same issue
brew install pgloader brew install --HEAD pgloader and building from source doesnt work
Monterey 12.0.1 / Intel i7
Any workarounds or fixes?

This doesn't seem to be working for me. I'm on M1 Pro & using version 3.6.3
The make pgloader command is stuck forever & doesn't make any progress

It making and getting stuck with no feedback seems odd if you just skim through the makefile. Do you have some of the additional requirements? Like sbcl?

Oh thanks! I missed a couple of dependencies, the installation happened correctly! 👍

However, running ./build/bin/pgloader --help is stuck forever

@szynkowskij
Copy link

Same issue
brew install pgloader brew install --HEAD pgloader and building from source doesnt work
Monterey 12.0.1 / Intel i7
Any workarounds or fixes?

This doesn't seem to be working for me. I'm on M1 Pro & using version 3.6.3
The make pgloader command is stuck forever & doesn't make any progress

It making and getting stuck with no feedback seems odd if you just skim through the makefile. Do you have some of the additional requirements? Like sbcl?

Oh thanks! I missed a couple of dependencies, the installation happened correctly! 👍

However, running ./build/bin/pgloader --help is stuck forever

The only time I had some similar problems of it working very odd was when I was able to make it and create the binary, but I had a mixture of x86 and aarch64 required libs. I made sure to remove my x86 version of homebrew and all of its libs that came from a timemachine backup when I upgraded to my M1. That would be my only other guess at your problems without it giving any feedback.

pgloader does hang every once and a while for me on my mac, but that's usually somewhere in the process of loading large files.

@adimyth
Copy link

adimyth commented Apr 28, 2022

Same issue
brew install pgloader brew install --HEAD pgloader and building from source doesnt work
Monterey 12.0.1 / Intel i7
Any workarounds or fixes?

This doesn't seem to be working for me. I'm on M1 Pro & using version 3.6.3
The make pgloader command is stuck forever & doesn't make any progress

It making and getting stuck with no feedback seems odd if you just skim through the makefile. Do you have some of the additional requirements? Like sbcl?

Oh thanks! I missed a couple of dependencies, the installation happened correctly! 👍
However, running ./build/bin/pgloader --help is stuck forever

The only time I had some similar problems of it working very odd was when I was able to make it and create the binary, but I had a mixture of x86 and aarch64 required libs. I made sure to remove my x86 version of homebrew and all of its libs that came from a timemachine backup when I upgraded to my M1. That would be my only other guess at your problems without it giving any feedback.

pgloader does hang every once and a while for me on my mac, but that's usually somewhere in the process of loading large files.

It is acting in a non-deterministic fashion. I tried building again from source, however running make pgloader gets stuck forever

@panayao
Copy link

panayao commented Jun 8, 2023

pgloader.zip
For M1/M2 compiled on macOS Monterey 12.6.3

@panayao
Copy link

panayao commented Jun 8, 2023

LOAD DATABASE
     FROM      mysql://username:password@mysql.db.url/mydb
     INTO postgresql://postgres:password@postgress.db.url/postgres

 WITH include drop, create tables, create indexes, reset sequences,
      workers = 1, concurrency = 1,
      multiple readers per thread, rows per range = 50000

  SET PostgreSQL PARAMETERS
      maintenance_work_mem to '128MB',
      work_mem to '12MB',
      search_path to 'postgres, public, "$user"'

  SET MySQL PARAMETERS
      net_read_timeout  = '120',
      net_write_timeout = '120'

 CAST type bigint when (= precision 20) to bigserial drop typemod,
      type date drop not null drop default using zero-dates-to-null,
      type year to integer;

I was able to get it to work but only if workers = 1 otherwise seems to crash

@philharmonie
Copy link

philharmonie commented Jul 31, 2023

I'm stuck as well on the same topic:

debugger invoked on a SIMPLE-ERROR in thread
#<THREAD "main thread" RUNNING {700B600003}>:
  Problem running initialization hook #<FUNCTION OPEN-FOREIGN-LIBS>:
  Unable to load foreign library (LIBCRYPTO).
  Error opening shared object "/usr/local/Cellar/openssl@1.1/1.1.1m/lib/libcrypto.1.1.dylib":
  dlopen(/usr/local/Cellar/openssl@1.1/1.1.1m/lib/libcrypto.1.1.dylib, 0x000A): tried: '/usr/local/Cellar/openssl@1.1/1.1.1m/lib/libcrypto.1.1.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/Cellar/openssl@1.1/1.1.1m/lib/libcrypto.1.1.dylib' (no such file), '/usr/local/Cellar/openssl@1.1/1.1.1m/lib/libcrypto.1.1.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')).

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE] Skip this initialization hook.
  1: [ABORT   ] Exit from the current thread.

(SB-INT:CALL-HOOKS "initialization" (#<FUNCTION OPEN-FOREIGN-LIBS>) :ON-ERROR :ERROR)

I tried to compile it myself (without success) and I tried to use the build provieded by @panayao without success.

I am using openssl@3 and I don't know why it is trying so use 1.1 and how to work around that.

I tied

brew unlink openssl@3
Unlinking /opt/homebrew/Cellar/openssl@3/3.1.1_1... 5548 symlinks removed.
❯ brew link openssl@1.1
Linking /opt/homebrew/Cellar/openssl@1.1/1.1.1u... 3999 symlinks created.
❯ echo 'export PATH="/opt/homebrew/opt/openssl@1.1/bin:$PATH"' >> ~/.zshrc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests