Optional isolated environment

[fmoehler]

This is the same as PR #663 . The creator of the PR is currently OOO. Therefore this PR is opened.

The bosh-cli wipes parts of the environment provided by the host-system which are in some cases essential for compilation of bosh-releases. This PR follows a conservative fix-approach by adding an option to opt-out of this default. This can (and maybe should) be extended by a fix of the “isolation” from the host-environment. For more details, see #660

[anshrupani]

looks good

[aramprice]

I'd prefer to keep this complexity out of the bosh cli since the issue this appears to be addressing could also be side-stepped by running the bosh CLI in a docker container.

Perhaps there is some broader issue I'm missing here though?

Regarding the code changes, it seems like there are a number of additions, specifically for nix, beyond a "don't isolate my environment" flag, I'm wondering if those need to be in the bosh-cli repo?

Regarding the feature I am concerned that allowing more variation in the local environment will increase the surface area we will have to consider when triaging issues with the CLI.

[aramprice]

Perhaps a default value of "/usr/local/bin:/usr/bin:/bin:/sbin" should remain here and we allow an override with something like BOSH_CPI_COMAND_PATH.

[aramprice]

I think there are two core changes lurking in this PR

1. Allowing the current ENV to be passed to the CPI command

• My inclination is for this logic to live exclusively in the bosh-cli codebase, and not rely on the bosh-utils' system.execCmdRunner.buildComplexCommand() handling of cmd.UseIsolatedEnv at all.

It seems like this change alone might be sufficient for the requestors needs.

2. Allowing the the PATH value passed to the CPI to be overridden

• This could be handled by using BOSH_CPI_COMAND_PATH or defaulting to the existing value. However this is accomplished the CLI's default behavior should be the same (e.g. PATH=/usr/local/bin:/usr/bin:/bin:/sbin).

This might be a nice additional change if there is a need for a mostly isolated ENV except with modifications to PATH.

The other nix specific files don't belong in this repo.

Side note: bosh-cli appears to be the only project in the cloudfoundry org which sets system.Command.UseIsolatedEnv perhaps this capability could be removed if we handle the "isolated or not" logic into the bosh-cli?

[FloThinksPi]

@fmoehler @aramprice
indeed there are quite some files not relevant here imho. They should be removed like the nix files and envrc etc.
Also i think removal of some code parts is not neccecary e.g. the BOSH_CPI_USE_ISOLATED_ENV variable seems to be removed here. Dont know if thats intended this variable is also not documented in the bosh docs.

broader issue I'm missing here though?

The broader issue as far as i remember is that the isolation feature in case of create-release is somewhat pointless on systems other than ubuntu. It basically boils down to bosh-cli does not accept the environment variable the OS gives it - however then it proceeds calling OS packages (gcc etc.) and fails either not finding those as it looks not in the path the OS told the process to look in but in a hardcoded one. Or It cannot find the proper libraries development headers when compiling with GCC since it also does not pass the information the OS has given via ENV variables to the GCC process.

In this state afaik the current create-release with enabled isolation(default?) does not work on every linux/unix system that has all the needed tools/libraries installed, it just works on systems that have the tools installed how ubuntu places them into its filesystem specifically.

One way would also be to state that just ubuntu is supported and one should run this inside a ubuntu container/vm - however compared to whats needed to make it work on all unix systems it might be easy to enable all linux/unix systems when holding on to basic POSIX concepts - but the exclusion exists for windows it seems already #626

However now looking into the code i saw there is also BOSH_CPI_USE_ISOLATED_ENV - maybe it can be used to disable isolation of env entirely then building a bosh release would also work on a non ubuntu linux as bosh and all build tools behind it would respect the OS env. WDYT ?

Also disabling that feature and using all OS Envs would be ok as a solution maybe BOSH_CPI_USE_ISOLATED_ENV does this need to test that maybe.

[jpalermo]

We talked about this at the FIWG meeting.

We think a good direction is to just always enable the isolated environment within the bosh cli. Even removing the current flag that allows you to disable it. And then have the CLI pass in the full set of ENV variables that a CPI might need to run (PATH, LIBRARY_PATH, etc)

This might cause some breakages, but we can just deal with those if they arise.