cli defaults override new-build cabal.project settings

[hvr]

It appears the fix in #4657 wasn't enough.

I was looking into why setting

allow-boot-library-installs: True

in a cabal.project file would have no effect, and it turns out it's because in rebuildProjectConfig the cli-flags have highest priority, due to

    return (projectConfig <> cliConfig, localPackages)

however, in #4657 we deliberately injected applyDefaults so the cliConfig constructed via

    cliConfig = commandLineFlagsToProjectConfig
                  globalFlags configFlags configExFlags
                  installFlags haddockFlags

already has those defaults.

And here's a dumped version of what cliConfig looks like for a simple invocation of cabal new-build w/o any CLI arguments:

ProjectConfig{projectPackages = [], projectPackagesOptional = [],
              projectPackagesRepo = [], projectPackagesNamed = [],
              projectConfigBuildOnly =
                ProjectConfigBuildOnly{projectConfigVerbosity = NoFlag,
                                       projectConfigDryRun = Flag False, projectConfigOnlyDeps = Flag False,
                                       projectConfigSummaryFile = [], projectConfigLogFile = NoFlag,
                                       projectConfigBuildReports = Flag NoReports,
                                       projectConfigReportPlanningFailure = Flag False,
                                       projectConfigSymlinkBinDir = NoFlag, projectConfigOneShot = Flag False,
                                       projectConfigNumJobs = NoFlag, projectConfigKeepGoing = Flag False,
                                       projectConfigOfflineMode = Flag False, projectConfigKeepTempFiles = Flag False,
                                       projectConfigHttpTransport = NoFlag, projectConfigIgnoreExpiry = NoFlag,
                                       projectConfigCacheDir = NoFlag, projectConfigLogsDir = NoFlag,
                                       projectConfigStoreDir = NoFlag},
              projectConfigShared =
                ProjectConfigShared{projectConfigDistDir = NoFlag,
                                    projectConfigConfigFile = NoFlag, projectConfigProjectFile = NoFlag,
                                    projectConfigHcFlavor = NoFlag, projectConfigHcPath = NoFlag,
                                    projectConfigHcPkg = NoFlag,
                                    projectConfigHaddockIndex = Flag "$datadir/doc/$arch-$os-$compiler/index.html",
                                    projectConfigRemoteRepos = [], projectConfigLocalRepos = [],
                                    projectConfigIndexState = NoFlag, projectConfigConstraints = [],
                                    projectConfigPreferences = [], projectConfigCabalVersion = NoFlag,
                                    projectConfigSolver = Flag AlwaysModular, projectConfigAllowOlder = Nothing,
                                    projectConfigAllowNewer = Nothing, projectConfigMaxBackjumps = Flag 2000,
                                    projectConfigReorderGoals = Flag (ReorderGoals False),
                                    projectConfigCountConflicts = Flag (CountConflicts True),
                                    projectConfigStrongFlags = Flag (StrongFlags False),
                                    projectConfigAllowBootLibInstalls = Flag (AllowBootLibInstalls False),
                                    projectConfigPerComponent = Flag True,
                                    projectConfigIndependentGoals = Flag (IndependentGoals False)},
              projectConfigProvenance = fromList [],
              projectConfigLocalPackages =
                PackageConfig{packageConfigProgramPaths = MapLast{getMapLast = fromList []},
                              packageConfigProgramArgs = MapMappend{getMapMappend = fromList []},
                              packageConfigProgramPathExtra = [], packageConfigFlagAssignment = [],
                              packageConfigVanillaLib = NoFlag, packageConfigSharedLib = NoFlag,
                              packageConfigStaticLib = NoFlag, packageConfigDynExe = NoFlag,
                              packageConfigProf = NoFlag, packageConfigProfLib = NoFlag,
                              packageConfigProfExe = NoFlag, packageConfigProfDetail = NoFlag,
                              packageConfigProfLibDetail = NoFlag, packageConfigConfigureArgs = [],
                              packageConfigOptimization = NoFlag, packageConfigProgPrefix = NoFlag,
                              packageConfigProgSuffix = NoFlag, packageConfigExtraLibDirs = [],
                              packageConfigExtraFrameworkDirs = [], packageConfigExtraIncludeDirs = [],
                              packageConfigGHCiLib = NoFlag, packageConfigSplitObjs = NoFlag,
                              packageConfigStripExes = NoFlag, packageConfigStripLibs = NoFlag,
                              packageConfigTests = NoFlag, packageConfigBenchmarks = NoFlag,
                              packageConfigCoverage = NoFlag, packageConfigRelocatable = NoFlag,
                              packageConfigDebugInfo = NoFlag, packageConfigRunTests = NoFlag,
                              packageConfigDocumentation = Flag False,
                              packageConfigHaddockHoogle = Flag False, packageConfigHaddockHtml = Flag False,
                              packageConfigHaddockHtmlLocation = NoFlag,
                              packageConfigHaddockForeignLibs = Flag False,
                              packageConfigHaddockExecutables = Flag False,
                              packageConfigHaddockTestSuites = Flag False,
                              packageConfigHaddockBenchmarks = Flag False,
                              packageConfigHaddockInternal = Flag False, packageConfigHaddockCss = NoFlag,
                              packageConfigHaddockHscolour = Flag False,
                              packageConfigHaddockHscolourCss = NoFlag, packageConfigHaddockContents = NoFlag,
                              packageConfigHaddockForHackage = NoFlag},
              projectConfigSpecificPackage = MapMappend{getMapMappend = fromList []}}

As can be seen, all flags which have a definitive Flag ... constructor will effectively override cabal.project settings, and specifically

projectConfigAllowBootLibInstalls = Flag (AllowBootLibInstalls False),

was the setting I was looking for...

and it appears to me, that the default injection should instead happen in rebuildProjectConfig (which is called by establishProjectBaseContext)

/cc @alexbiehl @23Skidoo

[hvr]

After some more investigation, I think that defaults are supposed to be applied in

resolveSolverSettings :: ProjectConfig -> SolverSettings

resolveBuildTimeSettings :: Verbosity -> CabalDirLayout -> ProjectConfig -> BuildTimeSettings

PS: I'm currently testing applyFlagDefaults = id locally to see how bad this currently is.

[leftaroundabout]

Is this by any chance related to the issue I'm having here (with Cabal c480f69):

$ cabal install distributive-0.5.3 --flags="semigroups tagged"
Resolving dependencies...
Configuring distributive-0.5.3...
Failed to install distributive-0.5.3
Build log ( /home/sagemuej/.cabal/logs/ghc-7.10.2/distributive-0.5.3-1mEGskitVaeLQN6CJ7QtG8.log ):
cabal: Entering directory '/tmp/cabal-tmp-31691/distributive-0.5.3'
[1 of 1] Compiling Main             ( /tmp/cabal-tmp-31691/distributive-0.5.3/dist/setup/setup.hs, /tmp/cabal-tmp-31691/distributive-0.5.3/dist/setup/Main.o )
Linking /tmp/cabal-tmp-31691/distributive-0.5.3/dist/setup/setup ...
Configuring distributive-0.5.3...
setup: '--exact-configuration' was given, but the following flags were not
specified: FlagName "semigroups", FlagName "tagged"
cabal: Leaving directory '/tmp/cabal-tmp-31691/distributive-0.5.3'
cabal: Error: some packages failed to install:
distributive-0.5.3-1mEGskitVaeLQN6CJ7QtG8 failed during the configure step.
The exception was:
ExitFailure 1

(I really don't see why I should need to define those flags at all, as they're enabled by default. I don't recall ever needing to specify flags in old builds. But now the “flags were not
specified” error happens regardless of what --flags="..." I give cabal-install.)

[hvr]

@leftaroundabout what you're experience appears to be a totally different issue, which looks serious enough to deserve its own issue being filed...

[leftaroundabout]

Thanks, I did that.

[hvr]

@dcoutts confirmed that I'm on the right track regarding #4737 (comment)

...so we need to basically revert 1e90ae4 (#4422) & #4657 and instead start fixing/finishing the resolve* functions to kill any left-over partiality.

See also comments at

cabal/cabal-install/Distribution/Client/ProjectConfig/Types.hs

Lines 339 to 349 in 22d8e32

	-- Resolving configuration to settings
	--
	-- | Resolved configuration for the solver. The idea is that this is easier to
	-- use than the raw configuration because in the raw configuration everything
	-- is optional (monoidial). In the 'BuildTimeSettings' every field is filled
	-- in, if only with the defaults.
	--
	-- Use 'resolveSolverSettings' to make one from the project config (by
	-- applying defaults etc).
	--

[andreabedini]

I haven't checked the code but from comments above it looks like this was fixed.