Make install directory a config option

[hegner]

This PR refactors the settings for the install area into a separate install_area.py and makes the location of this area a configurable. Even though there is only one layout currently, the directory_layout has been made a config option as well.

[hegner]

@tgamblin - what would be needed to finish this one?
As I was touching plenty of files in this PR, I was flooded with warnings. Used some waiting time to tackle a few of those. Obviously this increases the risk for merge conflicts. But probably unavoidable quickly after introducing checks.

[mathstuf]

@hegner I'd recommend splitting out the style fixes. The conflict resolution train is always longer than you think :) .

[mamelara]

Any word on the status of this PR? NERSC would be very interested changing the install path.

[hegner]

@mamelara - I think @tgamblin should comment on the style fix issues. I could then do the necessary changes - if any - in no time.

[adamjstewart]

Out of curiosity, what happens when two users install Spack in their home directories and both set their installation directory to the same directory. Are the install database and locks stored in the installation directory or should I expect to see lots of "spack find: not installed, spack install: already installed, spack uninstall: not installed" kind of confusion?

Someday I hope to have multiple system admins capable of installing software with Spack at the same time. Since at least one of us would be actively developing Spack and changing git branches, we would need to have multiple Spack installations. But it would be annoying to have to have multiple installation directories.

[tgamblin]

The DB and its lock live in the installation directory ($root/.spack-db/index.yaml, $root/.spack-db/lock). And when we change to finer-grained locking and parallel installs, locks will be per-prefix.

[alalazo]

And when we change to finer-grained locking and parallel installs, locks will be per-prefix.

@tgamblin OT, just to share the experience. Beware of :

remove_dead_links(spack.stage_path)

in stage.py. It is bound to break things when you go parallel if multiple processes are using local storage on different nodes.

[tgamblin]

@hegner: I rebased this on develop and am adding some updates to make install.yaml into a top-level config.yaml. Can you check the box on this PR that says Allow edits from maintainers so that I can push updates here?

[adamjstewart]

install.yaml -> config.yaml sounds great to me! I'm very excited about moving towards generic Spack configurability.

[tgamblin]

You're going to like these modifications 😄

[hegner]

@tgamblin - here you have it

[tgamblin]

Thanks!

[justintoo]

@tgamblin @hegner Is the install directory self-contained? i.e. spack is also installed with no dependency on the spack source directory?

[tgamblin]

@justintoo: Once the install is done, it doesn't depend on the package.py files or the build stage at all. The build stage is actually removed by default after a successful install.

[tgamblin]

@hegner: are you sure you checked the box here? I can't seem to push. Maybe I'm doing something wrong.

[tgamblin]

Hm this is odd -- GitHub says "Add more commits by pushing to the install_environment branch on hegner/spack." but I can't' seem to do that.

[tgamblin]

Just closing and moving to #2152.

[hegner]

@tgamblin - I enabled it. Maybe the problem is that my repo isn't a direct fork of yours. But let's follow up on the new PR.