SCons not resolving some Fortran module dependencies

[alexksr]

Issue: SCons fails resolving Fortran module deps.
Importance: SCons failing for a standard Fortran project makes it unusable, questioning Fortran support.

I was asked to switch this SE discussion https://stackoverflow.com/questions/77681762/scons-mwe-for-a-generic-fortran-project to the SCons repo.

In this example SCons is not able to resolve standard dependencies between modules.
module0 depends on module1 and module2, but SCons issues gfortran -o src/module0.o -c src/module0.f90 which results in an error since module1.mod is missing.

There is a python script to set up the Fortran sources in the repo https://github.com/alexksr/SconsMWE. init1.py is a trivial example. For the sources generates by init2.py SCons shows the issue.

@mwichmann looked at the SCons code and pointed out this may be intentional. If this behaviour of SCons is really ment to be the default, please help me tell SCons the dependencies via Depends. Do I need the (quite stable) external package fortdepend to get the dependencies or is there an internal method?

[mwichmann]

"Intentional" doesn't necessarily mean correct, it just means to proceed with caution, if someone meant it that way :-)

You can get SCons to say what it's (not) doing here with a flag to turn on the appropriate warnings - "no dependency generated":

$ scons --warn=dependency
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...

scons: warning: No dependency generated for file: module1.mod (referenced by: src/module0.f90) -- file not found
File "/home/mats/.pyenv/versions/venv-system/bin/scons", line 8, in <module>

scons: warning: No dependency generated for file: module2.mod (referenced by: src/module0.f90) -- file not found
File "/home/mats/.pyenv/versions/venv-system/bin/scons", line 8, in <module>

gfortran -o src/module0.o -c -Jfortran_mods src/module0.f90
src/module0.f90:3:9:

    3 |     use module1
      |         1
Fatal Error: Cannot open module file 'module1.mod' for reading at (1): No such file or directory
compilation terminated.
scons: *** [src/module0.o] Error 1
scons: building terminated because of errors.

[bdbaddog]

To be clear, SCons doesn't fail resolving all fortran modules, but it is failing with your particular configuration.

[mwichmann]

We do have the (quite "mature") issue #2022 indicating that modules in a subdirectory have been a concern for a while (and something I was poking at earlier this year). Looking at this as time allows - it actually appears quite straightforward to fix, but that fix trips up several places in the test suite, so have to look at those to see if it's bad test assumptions or actually breaking other Fortran things - so it may not be so simple.

[alexksr]

I guess this behaviour is not related to the subdir problem #2022. I created an "flat" version of the source file layout with no subdirs in init3.py in the above mentioned repo and also bdbaddogs SConstruct file. It still gets the deps wrong.

[mwichmann]

It is and it isn't, which is a long story. Anyway, using the short SConstruct, here's the dependency tree:

$ orig/scripts/scons.py -n --tree=prune,linedraw -Q
└─┬.
  ├─SConstruct
  ├─┬fortran_mods
  │ ├─┬fortran_mods/module0.mod
  │ │ ├─src/module0.f90
  │ │ └─/bin/gfortran
  │ ├─┬fortran_mods/module1.mod
  │ │ ├─src/module1.f90
  │ │ └─/bin/gfortran
  │ ├─┬fortran_mods/module2.mod
  │ │ ├─src/module2.f90
  │ │ └─/bin/gfortran
  │ └─┬fortran_mods/util_module.mod
  │   ├─src/utils/util_module.f90
  │   └─/bin/gfortran
  ├─┬main
  │ ├─┬src/main.o
  │ │ ├─src/main.f90
  │ │ └─/bin/gfortran
  │ ├─┬src/module0.o
  │ │ ├─src/module0.f90
  │ │ └─/bin/gfortran
  │ ├─┬src/module1.o
  │ │ ├─src/module1.f90
  │ │ └─/bin/gfortran
  │ ├─┬src/module2.o
  │ │ ├─src/module2.f90
  │ │ └─/bin/gfortran
  │ ├─┬src/utils/util_module.o
  │ │ ├─src/utils/util_module.f90
  │ │ └─/bin/gfortran
  │ └─/bin/gfortran
  └─┬src
    ├─src/main.f90
    ├──[src/main.o]
    ├─src/module0.f90
    ├──[src/module0.o]
    ├─src/module1.f90
    ├──[src/module1.o]
    ├─src/module2.f90
    ├──[src/module2.o]
    └─┬src/utils
      ├─src/utils/util_module.f90
      └──[src/utils/util_module.o]

and with my working patch (which builds):

$ work/scripts/scons.py -n --tree=prune,linedraw -Q
└─┬.
  ├─SConstruct
  ├─┬fortran_mods
  │ ├─┬fortran_mods/module0.mod
  │ │ ├─src/module0.f90
  │ │ ├─┬fortran_mods/module1.mod
  │ │ │ ├─src/module1.f90
  │ │ │ └─/bin/gfortran
  │ │ ├─┬fortran_mods/module2.mod
  │ │ │ ├─src/module2.f90
  │ │ │ └─/bin/gfortran
  │ │ └─/bin/gfortran
  │ ├──[fortran_mods/module1.mod]
  │ ├──[fortran_mods/module2.mod]
  │ └─┬fortran_mods/util_module.mod
  │   ├─src/utils/util_module.f90
  │   └─/bin/gfortran
  ├─┬main
  │ ├─┬src/main.o
  │ │ ├─src/main.f90
  │ │ ├──[fortran_mods/module0.mod]
  │ │ ├──[fortran_mods/module1.mod]
  │ │ ├──[fortran_mods/module2.mod]
  │ │ ├──[fortran_mods/util_module.mod]
  │ │ └─/bin/gfortran
  │ ├─┬src/module0.o
  │ │ ├─src/module0.f90
  │ │ ├──[fortran_mods/module1.mod]
  │ │ ├──[fortran_mods/module2.mod]
  │ │ └─/bin/gfortran
  │ ├─┬src/module1.o
  │ │ ├─src/module1.f90
  │ │ └─/bin/gfortran
  │ ├─┬src/module2.o
  │ │ ├─src/module2.f90
  │ │ └─/bin/gfortran
  │ ├─┬src/utils/util_module.o
  │ │ ├─src/utils/util_module.f90
  │ │ └─/bin/gfortran
  │ └─/bin/gfortran
  └─┬src
    ├─src/main.f90
    ├──[src/main.o]
    ├─src/module0.f90
    ├──[src/module0.o]
    ├─src/module1.f90
    ├──[src/module1.o]
    ├─src/module2.f90
    ├──[src/module2.o]
    └─┬src/utils
      ├─src/utils/util_module.f90
      └──[src/utils/util_module.o]

[alexksr]

Thanks for all the effort so far. There seems to be a lot of things going on internally.

As a plain user I'm interested in any workaround. Which version (your work/scripts/scons.py) do you use and is it available to the public? Or is there a way to tell SCons explicit dependencies via Depends or Requires? It is no problem to generate those deps with some external program, I just got stuck telling SCons the deps via its API with my "long version SConstruct" file .

[mwichmann]

In theory, yes - that's what Depends is for, to fill in things that can't be intuited. My experience is, it sometimes is a bit hard to get right.

[mwichmann]

and the work is not secret, just the current bit isn't up anywhere; it builds off pending #4380, which tries to fix problems with handling of a Fortran (and D) module directory. I'll put up a further patch presently, that's not, as noted, fully debugged/vetted.

[mwichmann]

fortran-modules.patch.gz

Here's the temporary patchfile (gzipped because of silly github file-suffix rules)

[bdbaddog]

Also I've forked your repo, and see SConstruct.simple2 for your init2 example
https://github.com/bdbaddog/SconsMWE

Probably could be more simplified. But this seems to work.

SConstruct.simple2 also seems to work for your init3 sample

[bdbaddog]

basically you want the object for the source which uses a module to depend on the .mod generated from the source providing that module.

@mwichmann's gzip'd patch should fix SCons to do the right thing with my SConstruct.simple, but if for some reason you can't patch your scons install, then SConstruct.simple2, should likely work.
Note that the logic in your SConstruct didn't seem to work if there are more than one main programs.

[bdbaddog]

One additional note here.
If you have to use Depend() to make your build work, 99.9% of the time either you're doing something wrong, or you may have found a bug with SCons, so Depend() would be a workaround until there's a fix.

[alexksr]

I tested the .simple2 script, also in some different contexts. It's working like a charm. For portability and compatibility reasons I don't want to stick to a patch.
Thanks for the time you put in this!

[bdbaddog]

Glad to be of help. Likely a fix will make it into the next release. Though no fixed timeline for the next release.

If you like to and if you can, please consider supporting @mwichmann and/or I through Github Sponsors. https://github.com/sponsors/bdbaddog and/or https://github.com/sponsors/mwichmann

And also thank for providing an easy to use reproducer for the issue. It's rare we get one as useful as the one you've provided.

[mwichmann]

Glad you can proceed; meanwhile this clearly identified an issue in how .mod files should be handled, so a fix for that will appear in some form in the next release.