./configure + homebrew: recommend the script .homebrew-build-env

[jhpalmieri]

When homebrew is detected, the "hints" at the end of running ./configure should suggest running the script .homebrew-build-env.

[Marking it as critical because it has the potential of reducing the number of support questions for release 9.1]

CC: @mkoeppe @dimpase @dcoudert

Component: build: configure

Author: Matthias Koeppe

Branch/Commit: ec64fb6

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/29504

[jhpalmieri]

comment:1

Questions:

• should .homebrew-build-env have a #!... line at the top?
• with zsh, the new default shell on OS X, . .homebrew-build-env fails, but source .homebrew-build-env works. Do we always suggest source .homebrew..., or is there a reason to suggest . .homebrew..., for example when using bash? With zsh:

% . .homebrew-build-env     
.: no such file or directory: .homebrew-build-env

I guess in zsh, the correct invocation with . would be . ./.homebrew-...: you have to specify the path.

[mkoeppe]

comment:2

Replying to @jhpalmieri:

Questions:

• should .homebrew-build-env have a #!... line at the top?
• with zsh, the new default shell on OS X, . .homebrew-build-env fails, but source .homebrew-build-env works. Do we always suggest source .homebrew..., or is there a reason to suggest . .homebrew..., for example when using bash? With zsh:

% . .homebrew-build-env     
.: no such file or directory: .homebrew-build-env

I guess in zsh, the correct invocation with . would be . ./.homebrew-...: you have to specify the path.

Good idea. We should recommend source ./.homebrew-build-env then.

No #! because it must be sourced, not executed

[mkoeppe]

comment:3

References:

_ environment variables - What is the difference between '.' and 'source' in shells? - Unix & Linux Stack Exchange
https://unix.stackexchange.com/questions/58514/what-is-the-difference-between-and-source-in-shells

bash - running script with ". " and with "source " - Unix & Linux Stack Exchange
https://unix.stackexchange.com/questions/17815/running-script-with-and-with-source

[mkoeppe]

Branch: u/mkoeppe/__configure___homebrew__recommend_the_script__homebrew_build_env

[mkoeppe]

New commits:

1bb30bf

build/bin/sage-print-system-package-command [homebrew]: Advertise .homebrew-build-env

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

Commit: 1bb30bf

[jhpalmieri]

comment:6

Thank you for working on this, but this only prints the message if there are uninstalled homebrew packages. Otherwise, this code gets executed before the message has a chance to be printed:

if [ -z "$system_packages" ]; then
    # No command needed to install nothing
    exit 0
fi

[mwageringel]

Replying to @jhpalmieri:

When homebrew is detected, the "hints" at the end of running ./configure should suggest running the script .homebrew-build-env.

If one sources this file, one would always need to make sure that it is sourced whenever one runs make or sage -b, right? If so, the hint should mention this. Still, I would probably forget about it sooner or later and end up with a broken build.

[jhpalmieri]

comment:8

Replying to @mwageringel:

Replying to @jhpalmieri:

When homebrew is detected, the "hints" at the end of running ./configure should suggest running the script .homebrew-build-env.

If one sources this file, one would always need to make sure that it is sourced whenever one runs make or sage -b, right? If so, the hint should mention this. Still, I would probably forget about it sooner or later and end up with a broken build.

Are you suggesting deleting the file altogether, or automatically running it every time?

[mwageringel]

comment:9

Replying to @jhpalmieri:

Are you suggesting deleting the file altogether, or automatically running it every time?

I am not suggesting to delete it, as it is used by the tox system, but, if my understanding is correct, then it certainly seems less error-prone if Sage could be configured to source it during the build process.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

ec64fb6

configure: Always output the homebrew env hint on homebrew

[sagetrac-git]

Changed commit from 1bb30bf to ec64fb6

[mkoeppe]

comment:11

How about this version?

configure: notice: the following SPKGs did not find equivalent system packages: cbc gp2c libsemigroups ninja_build pari_elldata pari_galpol pari_nftables pari_seadata perl_term_readline_gnu
checking for the package system in use... homebrew
configure: hint: installing the following system packages is recommended and may avoid building some of the above SPKGs from source:
configure:   $ brew install ninja
# Afterwards: 
# To automatically take care of homebrew messages regarding 
# keg-only packages for the current shell session:
# $ source /Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/.homebrew-build-env
# Add this to your shell profile if you want it to persist between shell sessions.
configure: After installation, re-run configure using:
configure:   $ ./config.status --recheck && ./config.status

[mkoeppe]

comment:12

Replying to @mwageringel:

... if Sage could be configured to source it during the build process.

I would like to keep all interactions with the system package manager on the level of hints issued to the user.
The user should be in charge of their environment.

[jhpalmieri]

comment:13

My initial reaction is that this won't work, since I have multiple Sage builds in different directories, but then I realized that the same script .homebrew-build-env works in all of those directories at the same time. So this would be fine with me.

The other option would be that running source .homebrew-build-env once would change the configuration so that future builds within that directory would automatically have PKG_CONFIG_PATH, etc., keep the same values. I think that's what @mwageringel was suggesting.

[mkoeppe]

comment:14

Replying to @jhpalmieri:

The other option would be that running source .homebrew-build-env once would change the configuration so that future builds within that directory would automatically have PKG_CONFIG_PATH, etc., keep the same values. I think that's what @mwageringel was suggesting.

OK, I think this would actually be best handled via #29507 rather than making it homebrew-specific. Something for 9.2.

[jhpalmieri]

comment:15

I would suggest waiting until #29507 is resolved, instead of adding this message now and then possibly changing it after #29507. (But if others feel strongly that it should go in now, I won't object. I like the wording in comment:11.)

[mkoeppe]

comment:16

The only thing that would change with the #29507 is the last sentence; and it won't be a problem if users have already put this line in their shell profiles.

So I would be in favor of getting this into 9.1. Otherwise, this will be an FAQ on sage-support

[jhpalmieri]

comment:19

Replying to @mkoeppe:

The only thing that would change with the #29507 is the last sentence; and it won't be a problem if users have already put this line in their shell profiles.

So I would be in favor of getting this into 9.1. Otherwise, this will be an FAQ on sage-support

Okay. I think it will still be an FAQ, since people are not in the habit of running ./configure before running make.

I still don't see the message at the end of ./configure, though, when I have installed all of the recommended packages:

$ ./configure
...
configure: notice: the following SPKGs did not find equivalent system packages: cbc gp2c libsemigroups pari_elldata pari_galpol pari_nftables pari_seadata perl_term_readline_gnu
checking for the package system in use... homebrew
configure: No equivalent system packages for homebrew are known to Sage
$

[dcoudert]

comment:20

I tried and saw the message at the end of ./configure.

By the way, for some reason I made a make distclean, and its first action was to run ./configure... and again after typing make.

[mwageringel]

comment:21

Replying to @mkoeppe:

Replying to @mwageringel:

... if Sage could be configured to source it during the build process.

I would like to keep all interactions with the system package manager on the level of hints issued to the user.
The user should be in charge of their environment.

Ok, fair enough. The new message is better indeed. Adding the source statement to one's shell profile has the potential to cause problems when compiling things outside of Sage I believe, as there must be a reason these packages are keg-only, so hopefully only users who understand the consequences of that (not me) do that.

[mkoeppe]

comment:22

Replying to @dcoudert:

By the way, for some reason I made a make distclean, and its first action was to run ./configure... and again after typing make.

We have two tickets for reducing the number of times that configure gets run: #29460 (Run configure less often) and #29310 ("make distclean" should not run "./configure")

[dcoudert]

comment:23

Thank you so much ! :))

[mkoeppe]

comment:24

Replying to @jhpalmieri:

I still don't see the message at the end of ./configure, though, when I have installed all of the recommended packages:

Right, sorry, there's one more place to fix

[dimpase]

comment:25

what is this gettext in for l in gettext; do in .homebrew-build-env ?
I'm confused.

[jhpalmieri]

comment:26

Replying to @dimpase:

what is this gettext in for l in gettext; do in .homebrew-build-env ?
I'm confused.

The loop runs once with l equal to "gettext", right? I assume it's written like this in case more packages eventually need to be treated the same way.

[mkoeppe]

comment:27

Replying to @jhpalmieri:

Replying to @dimpase:

what is this gettext in for l in gettext; do in .homebrew-build-env ?
I'm confused.

The loop runs once with l equal to "gettext", right? I assume it's written like this in case more packages eventually need to be treated the same way.

That's right.

[mkoeppe]

comment:28

Replying to @jhpalmieri:

I still don't see the message at the end of ./configure, though, when I have installed all of the recommended packages:

$ ./configure
...
configure: notice: the following SPKGs did not find equivalent system packages: cbc gp2c libsemigroups pari_elldata pari_galpol pari_nftables pari_seadata perl_term_readline_gnu
checking for the package system in use... homebrew
configure: No equivalent system packages for homebrew are known to Sage
$

Wait... why do you want to see a message here? Everything is in order

[dimpase]

comment:29

Replying to @mkoeppe:

Replying to @jhpalmieri:

Replying to @dimpase:

what is this gettext in for l in gettext; do in .homebrew-build-env ?
I'm confused.

The loop runs once with l equal to "gettext", right? I assume it's written like this in case more packages eventually need to be treated the same way.

That's right.

thanks. That was too creative for me :-)

[mkoeppe]

comment:30

Replying to @jhpalmieri:

I think it will still be an FAQ, since people are not in the habit of running ./configure before running make.

Well, we could do #29316 (Require ./configure before make)...

At least we should make sure that ./configure appears in all build instructions. We changed some in #29496 (Update README.md and Installation Guide) already, but there are more

[jhpalmieri]

comment:31

Replying to @mkoeppe:

Wait... why do you want to see a message here? Everything is in order

That's a fair question. Maybe I don't need to see the message? Are we convinced that if ./configure finds everything, then all of Sage's packages will also find everything? (It turns out that I had run source .homebrew-... earlier in that shell session, which is why ./configure was happy.)

A minor point: when I see the message, I see

# Afterwards: 
# To automatically take care of homebrew messages regarding 
# keg-only packages for the current shell session:
# $ source ./.homebrew-build-env
# Add this to your shell profile if you want it to persist between shell sessions.

It would be nice if it gave an absolute path so I could cut and paste it into my shell profile. But this is not a big deal: someone who has been installing homebrew packages probably knows enough to change this. But would this change be helpful?

diff --git a/build/bin/sage-print-system-package-command b/build/bin/sage-print-system-package-command
index 7802d447df..ee3d3c0c5b 100755
--- a/build/bin/sage-print-system-package-command
+++ b/build/bin/sage-print-system-package-command
@@ -45,7 +45,7 @@ case $system:$command in
     homebrew*:install)
         echo "# To automatically take care of homebrew messages regarding "
         echo "# keg-only packages for the current shell session:"
-        [ -n "$SAGE_ROOT" ] || SAGE_ROOT=.
+        [ -n "$SAGE_ROOT" ] || SAGE_ROOT=`pwd`
         echo "# $ source $SAGE_ROOT/.homebrew-build-env"
         echo "# Add this to your shell profile if you want it to persist between shell sessions."
         ;;

[mkoeppe]

comment:32

Replying to @jhpalmieri:

A minor point: when I see the message, I see

# Afterwards: 
# To automatically take care of homebrew messages regarding 
# keg-only packages for the current shell session:
# $ source ./.homebrew-build-env
# Add this to your shell profile if you want it to persist between shell sessions.

It would be nice if it gave an absolute path so I could cut and paste it into my shell profile. But this is not a big deal: someone who has been installing homebrew packages probably knows enough to change this. But would this change be helpful?

Did you run bootstrap? There is already a change that does that

[jhpalmieri]

comment:33

Running bootstrap fixes it. When does that need to be run? The only information I see says to run it if ./configure doesn't exist.

[mkoeppe]

comment:34

Replying to @jhpalmieri:

Replying to @mkoeppe:

Wait... why do you want to see a message here? Everything is in order

That's a fair question. Maybe I don't need to see the message? Are we convinced that if ./configure finds everything, then all of Sage's packages will also find everything?

This strong statement is not true because of #29448 (configure finds libpng but matplotlib does not) and perhaps also #27205 (Sage ignores libs from pkgconfig).

But a weaker statement is true: The set of settings that I make in .homebrew-build-env is minimal so that configure finds everything.

[mkoeppe]

comment:35

Replying to @jhpalmieri:

Running bootstrap fixes it. When does that need to be run? The only information I see says to run it if ./configure doesn't exist.

You only need to run it on the ticket because the configure tarball is not current.

[jhpalmieri]

comment:36

Okay, let's proceed with this. Thank you for being so patient with all of my questions.

[jhpalmieri]

Reviewer: John Palmieri

[mkoeppe]

comment:38

Thank you!

[vbraun]

Changed branch from u/mkoeppe/__configure___homebrew__recommend_the_script__homebrew_build_env to ec64fb6