Changes to homebrew-cask installation behaviour

[vitorgalvao]

Changes to homebrew-cask installation behaviour

Depending on how long you’ve been following the development of homebrew-cask (henceforth HBC), you might’ve come across this great rant¹. It was the turning point in making the link stanza (precursor to app) a requirement to every cask. Prior to that, HBC tried hard to be smart about what to link, but we eventually concluded being smart doesn’t work — there are too many edge cases.

The end of linking, and the beginning of moving (reasoning)

We reached another such point with linking. What works for CLI apps does not work for GUI apps, and by sticking close to the homebrew model we’re bending apps to do what they’re not expecting, and only suffering from it. Issues we cannot fix continue to pop up, and copy as merely a preference or something to be done occasionally can’t solve them. We cannot, in addition to the inconsistency with pkgs being installed instead of linked, add yet one extra inconsistency where even apps won’t have a predictable behaviour. To put it bluntly, linking sucks for GUI apps. It doesn’t work and will never not be broken, as we can’t control what other developers do.

Linking isn’t even needed, if you think about it. What separation of concerns are we worried about? CLI tools should have the separation homebrew provides because some of their apps (like newer versions of OS-level tools) provide critical features that could break the entire system if overridden. That is not the case at all with GUI apps; the only apps we have that could break the system on install (pkgs) are already given free range. If you use HBC, there’s not even any reason to install a GUI app any other way, so why would that separation matter?

GUI apps aren’t the only suffering from the current behaviour, even. Other artifacts suffer from similar issues and some even required the introduction of a different kind of linking.

In addition, while having different versions of the same CLI app is many times desirable, that is a far from the norm with GUI apps. Having an upgrade that works on every app, even ones that auto-update, also becomes nonsensical.

We should emulate regular usage (i.e. what people do without HBC), instead of trying to perpetuate a new paradigm that is basically contradicting expectations. This, however, does not mean losing customisation options when it makes sense (like --appdir).

Every workaround we build is forcing apps to fit our model, and our model is bad. We’re building workaround on top of workaround, adding more code and work for something that will never work correctly.

From The Cathedral and the Bazaar: “you often don't really understand the problem until after the first time you implement a solution. The second time, maybe you know enough to do it right. So if you want to get it right, be ready to start over at least once”.

As with other changes in the past, we now understand this problem a bit better and will adjust our solution accordingly.

The end of linking, and the beginning of moving (technical)

First steps towards new behaviour:

1. Every artifact stanza except binary will move its target to the appropriate location, instead of being sym/hardlinked.
2. apps will be moved to /Applications by default.
3. :target will continue to work, but rename the artifact itself instead of a link.
4. The Caskroom (/opt/homebrew-cask/Caskroom/ by default) still needs to exist.
5. Versioned subdirectories of the Caskroom won’t exist anymore, though.

Explanations:

1. Explained in the reasonings above why most artifacts will be moved instead of linked. As for binaries being the exception, it’s simple: not only is it expected, it’s advantageous, recommended, and sometimes necessary. Many times a binary is linked from inside an app bundle and cannot simply be moved/copied from there and continue to work. In these cases, symlinking is the recommended (by the developer) course of action anyway, so we stay in line with the idea of emulating regular usage. It also makes it work similarly to other CLI tools, like installs from homebrew.
2. Self-explanatory. It’s what most users expect, and would be our default anyway.
3. Self-explanatory.
4. It will be needed to keep a few things:
   1. The originals from binary, when they’re standalone.
   2. Other things like uninstall scripts that come with some pkgs.
5. Explained in the reasonings above.

New stanzas/keys/flags to enhance these changes

1. auto_updates. Similarly to stage_only, it should be used when a value of true is needed. It means “the artifact this cask uses auto-updates, so even if the cask file updates, leave this alone when upgrading others”. This does not mean upgrade in general is no longer a desirable feature — it is, but for casks who do not do it themselves.
2. :required_location. Example: app 'FontForge.app', :required_location => '/Applications'. It means the artifact needs to be in a certain location to function correctly and as such will be moved there even if you have a different default location.
3. --force-appdir. Takes no arguments and forcibly overrides required_location.

Pseudo-code for :required_location, considering an app:

if (--appdir is unset)
  move_to /Applications
else
  if (cask has :require_location) and (required_location_target != wherever_appdir_is_set_to)
    print_warning "This app only works if installed in required_location_target, so it will be moved there"
    move_to required_location_target
  else
    move_to wherever_appdir_is_set_to
end

Doesn’t this mean a bunch of stuff might break?

In the short term, yes, it does, but it is the right decision. We understand we have a bunch of users that are using homebrew-cask daily, and ideally we could try to make this a smooth transition like in the past. However, we’re admittedly pre-alpha and setting transition strategies is an inglorious job (everything will be discarded in the end, after all) that puts on hold even the smallest of changes indefinitely. It is incredibly time consuming, has variable results (we still get a bunch of issues pertaining to it), and (above all) kills momentum. Furthermore, this is a big change and there’s really no good way to make this seamlessly.

In the end, we want to continue to move forward and improve HBC. Being afraid of changes in a pre-alpha would mean stagnation.

Are there any other big changes planned?

There are ideas, yes, but no exact plans yet. Those ideas still need to be discussed and most are concerned with the creation and maintenance of casks, not the usage of HBC.

---

¹ If you haven’t, I highly recommend it as piece of HBC’s history.

[adidalal]

Seems like a useful change, and might be timely as well - I plan to do a clean install with 10.11, so breaking changes are not a huge issue.

So does the auto_updates stanza remove the need for explicit versioned urls, or is there still a preference for versioned, shasum'd formulas?

Would an example of a formula where auto_updates: true then be an app like Spotify, as it manages its own updates (ie, any forumas that use version :latest & sha256 :no_check)

[vitorgalvao]

and might be timely as well

Well, depending on how long the implementation (and release) takes.

So does the auto_updates stanza remove the need for explicit versioned urls, or is there still a preference for versioned, shasum'd formulas?

For now, preference should be left with versioned urls. The overlap of apps that auto-update and apps that have unversioned urls isn’t clear (auto_update will make it so), so this is a discussion for later. The security provided by always having a version is desirable.

as it manages its own updates (ie, any forumas that use version :latest & sha256 :no_check)

This is incorrect. A cask having an unversioned url (i.e. it almost exclusively will use version :latest) is not guaranteed to auto-update, or vice-versa. The two are completely unrelated.

[szhu]

Is there a milestone, task list, thread, or wiki page where people can go check how much of this has been implemented and/or released?

[vitorgalvao]

Is there a milestone, task list, thread, or wiki page where people can go check how much of this has been implemented and/or released?

Not currently. Things should be released as issues as time goes on. It might actually take a few weeks for development of this to start, but this is the type of thing that should be announce early.

[scribblemaniac]

👍
Everything sounds good to me, I'm glad to hear we are moving forward in this direction for the most part.

If apps come in a folder with a bunch of other things (readmes, licenses, plugin directories, etc.) will those be moved to /Applciations as well? What about if they are required to be in the same directory as the application?

Should auto_update casks be maintained from version to version?

If an application downloads the latest update when necessary, but does not install it for the user, should it be considered auto_update? I know that Filezilla does something like this (or it used to, I haven't checked recently).

[vitorgalvao]

If apps come in a folder with a bunch of other things (readmes, licenses, plugin directories, etc.) will those be moved to /Applciations as well?

No. See point 4 of The end of linking, and the beginning of moving (technical) in the top post. Only what we already link will be moved, it’s one to one.

What about if they are required to be in the same directory as the application?

Than we use suite to move the directory. We’re already covered, there, and some casks were already changed, even.

Should auto_update casks be maintained from version to version?

Yes, so anyone installing it for the first time gets the latest version. Many (most?) of these have discoverable appcasts, though, which makes the job easier.

If an application downloads the latest update when necessary, but does not install it for the user, should it be considered auto_update?

No. Only the ones that can do it seamlessly should, because otherwise we can still provide a better experience.

[szhu]

If apps come in a folder with a bunch of other things (readmes, licenses, plugin directories, etc.) will those be moved to /Applciations as well? What about if they are required to be in the same directory as the application?

(The following is my own preference.) I don't like folders in the Applications folder unless it really is to organize a bunch of apps (Microsoft Office).

Suggestions for apps like Audacity and Eclipse that have a required folder structure:

• Place folder contents in /Library/Application Support/Audacity, and place an app symlink in /Applications/Audacity.app -> /Library/Application Support/Audacity/Audacity.app
• Place folder contents in /Library/Application Support/net.sourceforge.audacity, and place an app symlink in /Applications/Audacity.app -> /Library/Application Support/net.sourceforge.audacity/Audacity.app (bc sometimes apps have the same name)
• Place folder contents in /Applications/Audacity, and place an app symlink in /Applications/Audacity.app -> /Applications/Audacity/Audacity.app, and chflags hidden /Applications/Audacity (probably not a good idea bc hidden files)

Suggestions for apps with non-code supporting files:

• Place supporting files in /Library/Application Support/SomeApp (more consistent with the above)
• Place supporting files in SomeApp.app/Support, SomeApp.app/Extras, SomeApp.app/, etc. (more consistent with Mac expectation that trashing the app will pretty much uninstall it)
• Discard supporting files
• Leave supporting files in Caskroom

[vitorgalvao]

I like those ideas, @szhu. However, I’ll also say “definitely not for now”, and the first paragraph of the top post explains why. Those could introduce issues we’re not foreseeing, and this change is in part meant to fix things, not introduce new possible points of failure.

It doesn’t mean we can’t revisit those in the future, naturally, but for now the takeaway from this post should be emulate regular usage. Let’s nail that first, and only then see if being smart can pay off.

[szhu]

Okay. I'm assuming that means the following:

• For apps that require a folder structure, move the entire folder to /Applications or leave it in the Caskroom
• For apps that provide a .app with some documentation, discard the documentation or leave it in Caskroom

[vitorgalvao]

For apps that require a folder structure, move the entire folder to /Applications or leave it in the Caskroom

Move it, using suite.

For apps that provide a .app with some documentation, discard the documentation or leave it in Caskroom

Leave it in the caskroom.

Both those points were addressed in the previous post.

[markhuber]

You reference /Applications , currently all of my casks have defaulted to ~/Applications . Is this change also changing that behavior of utilizing a folder local to the user's home directory? (sorry if this was covered elsewhere)

[scribblemaniac]

Yes, it is changing that behavior. You will still be able to use ~/Applications if you want by using by using the --appdir flag.

[tjnycum]

Was there an open discussion of the pro's and con's of this somewhere that I've missed?

[vitorgalvao]

@markhuber:

sorry if this was covered elsewhere

It was all covered in the top post. @scribblemaniac’s explanation is correct.

@tjnycum Yes, there were multiple, including #3083 and #3888. I have a slight feeling from your last comment and this one you might be under the impression this might be a matter of user preference and voting. It’s not.

This is about fixing a broken model, and I’ve spent many months thinking about this solution and managing homebrew-cask always with that in mind, to understand how it’d affect it. Keeping the old system is unsustainable.

Every workaround we build is forcing apps to fit our model, and our model is bad. We’re building workaround on top of workaround, adding more code and work for something that will never work correctly.

Every time there’s a new problem our workarounds fail to address we have to discuss how to implement a new one. Those are becoming harder and harder, and many of them are impossible to solve without breaking experience. All of them are solved immediately by emulating regular usage.

As mentioned in the top paragraph of the top post, being smart failed, and as mentioned in a previous post, let’s first get things to work, and think about being smart later.

[voanhduy1512]

👍 i want this for years

[vrillusions]

After just quickly scanning 👍 on the auto_update. Right now I just don't install apps that I know do automatic updating assuming it would break things. Also the moving of apps I can see as a much needed change as I've had issues with some apps being linked which result in me again not using cask but installing it the "Old Fashioned Way".

This big changes can be annoying in the short term but it's good to figure these things out while the project is still relatively new than wait 10 years and have to deal with everything breaking

[kevinSuttle]

I'm interested in how this is going to work in a situation where a user has already installed an app from the Mac App Store. Will an overwrite prompt be given?

[mafredri]

@kevinSuttle I believe that was addressed in the previous comments. Since the app was not installed beforehand, and the file exists, a warning/error will be given and no overwriting will take place.

[kevinSuttle]

Actually, I just tested out Skitch. There was no prompt at all.

Succeeded in installing skitch
Success: 55 Fail: 0

Brewfile
cask 'skitch'
using https://github.com/Homebrew/homebrew-bundle

[kevinSuttle]

Was able to reproduce with several App Store/free apps.

Evernote
Alfred
etc.

[jawshooah]

@kevinSuttle There is no prompt. This is what happens if you already have an app installed:

$ brew cask install skitch
==> Downloading https://cdn1.evernote.com/skitch/mac/release/Skitch-2.7.8.zip
######################################################################## 100.0%
==> It seems there is already an App at '/Applications/Skitch.app'; not linking.
🍺  skitch staged at '/opt/homebrew-cask/Caskroom/skitch/2.7.8' (1863 files, 73M)

No failure is reported because nothing failed. The cask was downloaded, unpacked, and staged. We won't remove an existing non-cask installation.

[kevinSuttle]

I'm telling you it's happening on my system. Clean install of El Capitan.

I'm running brew bundle, btw. Maybe that's the difference.

[jawshooah]

homebrew-bundle is an external plugin, and not officially supported by homebrew-cask. I can't speak for what it does or doesn't do.

Have you double-checked your /Applications/Skitch.app to make sure that it is, indeed, a symlink to the Cask installation? If not, then everything is working as intended.

[jawshooah]

In any case, what you are reporting would be a bug with the existing behavior of homebrew-cask. If you determine that we are, in fact, overwriting app bundles with symlinks, please open a separate issue.

[kevinSuttle]

@jawshooah Help my understand why homebrew-bundle is an external plugin, if it's maintained by the homebrew org on GitHub. cc @MikeMcQuaid

In any event, what's the recommended way to batch install multiple casks?

[nathany]

@kevinSuttle See https://github.com/Homebrew/homebrew-bundle for issues with homebrew-bundle

[kevinSuttle]

Homebrew/homebrew-bundle#112

[jawshooah]

@kevinSuttle We are not Homebrew. Homebrew and Homebrew-cask are maintained by completely separate teams. Therefore, homebrew-bundle is external to us.

If you want to install multiple casks, a simple shell script would suffice:

brew cask install \
  skitch \
  evernote \
  alfred \
  ...

Or perhaps:

brew cask install skitch
brew cask install evernote
brew cask install alfred
...

Or even:

casks=(
  skitch
  evernote
  alfred
  ...
)

for cask in "${casks[@]}"; do
  brew cask install "$cask"
done

[kevinSuttle]

Ok, that makes sense. Thanks @jawshooah.

[satoProgrammer]

Why is it that after I brew cask install github-desktop that my GitHub Desktop.app shows in my ~/Applications folder as a symlinked app but shows in launchpad as an app that isn't symlinked?

[jiegec]

@satomusic Oh launchpad don't mind whether it is a symlink or a directory.

[satoProgrammer]

Then can I take that directory from launchpad and apply that to my /Application folder?

[vitorgalvao]

@satomusic Your problem has nothing to do with this issue. Lets please stay on-topic, it is big enough as it is. Either way, it’ll be irrelevant once this is up, so we won’t work on it.

Furthermore, your last post is an OS X question, not a homebrew-cask question, so you should resort to a website with that specialty, like Ask Different from Stack Exchange.

[szhu]

@vitorgalvao I recommend you post one more comment that sort of acts as an FAQ and then lock this thread. I think the FAQ should include answers to the following questions (my best-guess answers are in parens):

• What is this thread about? (HBC will be changing the way it works, and this is an announcement)
• Are these changes in effect yet? (No)
• When will these changes be in effect? (Not sure, but not now)
• Is there a timeline for that? (No)
• Why not? (Because this project is run by volunteers)
• How can people help out? (By contributing code)
• What if someone has a question regarding this issue? (1. Check that it's actually related to this issue 2. Check that the answer isn't in this thread 3. If neither, open a new issue)

[vitorgalvao]

@szhu I like that, thank you. Nothing further to add to your FAQ, so we can leave it as is.

I didn’t want to close this at the start because I wanted people to find holes in it, but it seems like the idea as it stands is pretty solid and agreed upon, so yes, lets stop diverging further since comments have been becoming less relevant.

[vitorgalvao]

The first and most important part of this change is now very close to being implemented.

If you’re interested in helping test it, head over to #13966 and follow the instructions at the bottom.

[vitorgalvao]

Getting really close, now, so if you had some things you wanted to do that are dependent on this change, now is the time.

[vitorgalvao]

And merged.

There are still a few things that need to be done in order to close this issue, but the biggest user-facing change is now present.

[reitermarkus]

@vitorgalvao, what exactly still needs to happen for this to be closed? The only thing I'm seeing that is not yet implemented is :required_location and --force-appdir, which kind of contradict themselves.

Why would you be able to override a required location? That would mean the location isn't actually required.

[vitorgalvao]

@reitermarkus Versioned subdirectories of the Caskroom won’t exist anymore is also yet to be implemented.

As for --force-appdir, I do not remember my reasoning there, and I guess it could be stricken (not required_localtion:, though, naturally).

[jawshooah]

I suggested --force-appdir in #13201 (comment), mostly because I think the user should have final say in where their stuff is installed, and may have good reason for overriding the developer's stated location requirements.

We should absolutely print a warning when the user overrides required_location though, so they have no grounds to blame us for a broken installation.

[reitermarkus]

@vitorgalvao, I think we can close this in favour of #12858.

[vitorgalvao]

@reitermarkus Agreed. At this point other issues should take care of whatever is missing here.