Issue 1242 sphinx apidoc

[ltalirz]

Fix #1255

• Autogenerate complete API documentation of aiida using sphinx-apidoc

[ltalirz]

@giovannipizzi It's already progressed significantly (we are down to 3 warnings left to be resolved - thanks to @sphuber !).

What remains is to decide what to do with the old, manually generated API documentation.
Most of the sources/*/dev.rst files just contain a list of automodule/class/... statements for sub-selection of the code. In my view, these files are replaced adequately by the new api doc and should be deleted:

Delete:

• cmdline/dev.rst
• djsite/dev.rst
• execmanager/dev.rst
• parsers/dev.rst
• scheduler/dev.rst
• sqlalchemy/dev.rst
• testing/dev.rst
• tools/dev.rst
• work/dev.rst
• developer_guide/control/dev.rst
• querying/querybuilder/dev.rst

Others contain either a bit of text as well or some ordered selection, which might contain some thought. In my view, these could be integrated in other parts of the documentation, if needed.

To Decide:

• common/dev.rst : delete?
• orm/dev.rst : retain somehow?
• registrytools/dev.rst: delete?
• transport/dev.rst : retain
• import_export/dbexporters/dev.rst : retain the note
• import_export/dbimporters/dev.rst : delete?

@giovannipizzi Do you agree with this assessment?

[sphuber]

There are still issues with aiida.backends.sqlalchemy.migrations.env. It tries to access config from alembic.context which does not exist. Additionally, upon import of the file line 72 if context.is_offline_mode(): is executed which raises the following exception:

/home/sphuber/code/aiida/env/release/aiida-core/docs/source/apidoc/aiida.backends.sqlalchemy.migrations.rst:17: WARNING: autodoc: failed to import module u'aiida.backends.sqlalchemy.migrations.env'; the following exception was raised:
Traceback (most recent call last):
  File "/home/sphuber/.virtualenvs/aiida_release/local/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 551, in import_object
    __import__(self.modname)
  File "/home/sphuber/code/aiida/env/release/aiida-core/aiida/backends/sqlalchemy/migrations/env.py", line 72, in <module>
    if context.is_offline_mode():
  File "<string>", line 7, in is_offline_mode
  File "/home/sphuber/.virtualenvs/aiida_release/local/lib/python2.7/site-packages/alembic/util/langhelpers.py", line 99, in _name_error
    name, cls.__name__
NameError: Can't invoke function 'is_offline_mode', as the proxy object has not yet been established for the Alembic 'EnvironmentContext' class.  Try placing this code inside a callable.

I cannot fix this, I think @szoupanos should take a look at this

Finally once that warning disappears there is another problem: the autogenerated rst files say that the AiidaTestCase has the base unittest.case.TestCase which then raises an import error because sphinx cannot find it. We never inherit from this class explicitly but it is something that the apidoc assumes. Also here I have no clue how to fix it

[szoupanos]

@sphuber @ltalirz
OK, I fixed it and the commit was pushed to https://github.com/ltalirz/aiida_core/tree/issue_1242_sphinx-apidoc

I disabled sqla schema checks and alembic migration loading when compiling the docs since it doesn't make any sense to have such checks during documentation compilation - This is done for local documentation compilation but also for online documentation compilation.

[ltalirz]

@szoupanos Thanks!
Travis still reports the following warning:

  File "/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/django/conf/__init__.py", line 115, in __init__
    raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
ImproperlyConfigured: The SECRET_KEY setting must not be empty.

Any ideas how to best fix this?

[giovannipizzi]

Maybe the simplest thing is to create the SECRET_KEY before compiling the docs? (logic called by verdi setup)

[szoupanos]

Ooops, it seems that I pushed only half of my changes - Sorry!
I don't know if this is related to the SECRET_KEY issue but in any case let me commit everything.

[szoupanos]

@ltalirz
You were right. After adding the missing files the SECRET_KEY error disappeared.
Please let me know if Travis tests also pass.

[ltalirz]

@szoupanos Thanks, the issue is gone.
There is now a warning
/home/travis/build/ltalirz/aiida_core/docs/source/apidoc/aiida.backends.rst:8: WARNING: py:class reference target not found: unittest.case.TestCase

I wonder whether this points to missing dependencies in the travis build...

[szoupanos]

Yes, I saw it... And I don't get this warning when I compile it locally.
Moreover, after the compilation, I don't have any aiida.backends.rst or apidoc directory (maybe it is a temporary file & folder).

I have to have a closer look.

[ltalirz]

Well, after you compile the documentation you should have the source/apidoc directory, it's generated by sphinx-apidoc.
Thanks for looking into this - if you don't find the cause, let me know, it looks like something unrelated to aiida, so I can also look into it.