Initial implementation of astropy_helpers #1

embray · 2013-12-16T23:40:05Z

This starts to add the important pieces of astropy_helpers. It ports astropy.version_helpers and astropy.setup_helpers into the new package and modifies them to work in the new context, without access to the astropy package itself.

It also adds some initial infrastructure such as the ah_bootstrap.py script that other packages will use to bootstrap astropy_helpers when using it.

This should serve as an initial reference implementation for use with the first draft of the astropy_helpers APE. Rather than clone my fork of this directly, check out the pull request in astropy/astropy#1563. The branch I based that pull request off of pulls in this version of astropy_helpers (from my fork) as a submodule.

… from Astropy

…lpers and version_helpers modules from astropy; these are still heavily reliant on Astropy to work

…ule, mainly by copying over a few utility functions (some of these utilities are no longer used in astropy as a result--should we remove them?) Also removed any support for the astropy test and configuration frameworks, but that can be added back in later as needed...

…rs module. This enabled creating a version of generate_version_py that can work without having Astropy or any other support libraries installed, and that also provides support for projects that don't use git (though they lose revision numbers in their version strings until and unless we add support for other VCS (I do already have code to do this for SVN and HG should we wish to add it)).

…d at some point.

… module; previously this was returning the version module in astropy_helpers itself. This changes the get_debug_option function such that it requires the package name. Will probably refactor functions like that into a class that is already instatiated with basic info about the package being built, such as its name.

…t won't work with astropy_helpers in its current state. This can, and should be revisited once we have a better idea of what we're going to do about configuration.

…as well just used the dashed name explcitly

…rs in the body of the with statement, restore sys.stdout/err so that the exception is displayed properly. Also add a dummy flush method to _DummyFile as there is some code in distutils that calls sys.stdout.flush

…works similarly to setuptool' ez_setup.py in how it tries several different routes to find the astropy_helpers module and get it on sys.path. The default is to just import it from a submodule called astropy_helpers--this is the default that will be used when running the astropy repo, for example. But there are many other options. I've tested out all the major use cases, but once I add a test suite to astropy_helpers it will be nice to have tests for the boostrap script as well.

embray · 2013-12-16T23:43:47Z

astropy_helpers/setup_helpers.py

@@ -0,0 +1,1558 @@
+# Licensed under a 3-clause BSD style license - see LICENSE.rst


I intend to do some further refactoring of this module and break it up into some smaller modules. I also intend to reduce the amount of "boilerplate" that needs to be added to setup.py for all the features in here to work. I've already started a little of that work in a local branch.

Sounds good - that file was indeed getting a bit large!

astrofrog · 2013-12-17T10:07:12Z

@embray - thanks for all your work on this!

astrofrog · 2013-12-17T10:08:25Z

@embray - I know we are starting with the bare minimum, but wondering whether astropy.utils.release should be one of the next helpers to get moved over to this package?

embray · 2013-12-17T18:39:52Z

That might not be bad to move over either. Though I thinking in the long-term of moving away from zest.releaser. It's served me quite well, and I will probably continue to use it for many simpler projects. But the release process for Astropy has grown complex enough that I feel it would be difficult to fit within zest.releaser's framework, as it doesn't give quite enough control over the order in which certain things happen. I've been teasing myself with the idea of building up a release procedure using fabric. It would also start up a fresh copy of some standard VM for doing releases on, rather that doing releases directly from our desktop machines, so as to greatly improve reproducibility of releases. That's all long-term though. In the short-term astropy.utils.release does seem like it could go into astropy_helpers.

rhewett · 2013-12-17T19:19:52Z

Selfishly, it might be useful, as development progresses, to tag any area of the code that is Astropy specific (excepting grepable things). From a not-selfish perspective, it might help find areas that can be thought about more carefully.

(To make this less out of the blue, this references an offline conversation with @embray about turning this into a more general package at some point.)

astrofrog · 2014-01-13T19:30:40Z

ah_bootstrap.py

+        # earler versions of git returned 0 even if the requested submodule
+        # does not exist
+        log.debug('git submodule command failed '
+                  'unexpectedly:\n{0}'.format(sterr))


Typo - should be stderr

…tus. It appears that in some versions of git even a successful run can output some text to stderr; see astropy/astropy#1563 (comment)

…mber of needed 'bootstrap' scripts :)

… using __file__ may have made sense when this was used primarily by Astropy itself, but in the context of an affiliated package it makes no sense.

…tropy_helpers.tests--this will allow us to use the package template itself as our test project for testing astropy_helpers :)

…rom sdists to keep those small. Running the astropy_helpers tests should be of interest to developers only for now. Also make sure the testpackage isn't recursed into for py.test since we don't want to run *its* tests

…h a bug which was fixed in c3f841b

…' sandboxing--this way we can capture test coverage properly

astrofrog · 2014-05-16T17:44:46Z

@embray - if I got to the directory where the git command is being run, I get:

mac-robitaille2:package_template tom$ pwd
/private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/pytest-52/test_update_git_devstr0/package_template
mac-robitaille2:package_template tom$ ls
MANIFEST.in     ah_bootstrap.py cextern         ez_setup.py     packagename     setup.cfg       temp
README.rst      astropy_helpers docs            licenses        scripts         setup.py
mac-robitaille2:package_template tom$ git rev-parse HEAD
fatal: Not a git repository: ../../../.git/modules/astropy_helpers/tests/package_template
mac-robitaille2:package_template tom$ git log
fatal: Not a git repository: ../../../.git/modules/astropy_helpers/tests/package_template

weird, eh?

astrofrog · 2014-05-16T17:46:28Z

@embray - this might help explain it:

mac-robitaille2:package_template tom$ more .git
gitdir: ../../../.git/modules/astropy_helpers/tests/package_template

(because we aren't actually in a sub-directory of astropy_helpers)

astrofrog · 2014-05-16T17:47:19Z

The interesting thing is, why are you not seeing this error locally? In my case, it's working in a temporary directory that is no longer inside the astropy_helpers folder, so maybe it's something to do with that.

…ate the test failure on Travis

embray · 2014-05-16T19:12:09Z

That's for investigating this. That should help. I'll take a closer look...

embray · 2014-05-16T19:19:08Z

Okay, I understand what's going on here now. I'm still on git 1.7.5.x. In 1.7.8 they changed how the .git dir for submodules is handled. Instead it puts the .git dir contents for submodules under the top level repo's .git/, as ./git/modules/<submodule_name> as you posted above.

The .git for the submodule itself is just a plain file containing directives for where to find the actual .git dir.

I think this would be resolved if, instead of doing a shutil.copy of the package_template repo I just do a git clone of it. Let's give that a shot.

… to ensure the copy of package_template remains a valid git repo

embray · 2014-05-16T20:30:49Z

Ah, Python 3 support is broken in the sphinxexts. We never did de-2to3 them I don't think. Well that should be easy. So...close now...

embray · 2014-05-16T20:48:19Z

Aha, the tox.ini doesn't install Sphinx. That's why I was missing those tests when I ran them locally.

…d Python3.4 as well, and support for passing extra positional arguments to py.test

…aven't tested building actual docs with them yet.

astrofrog · 2014-05-16T21:36:48Z

@embray - how do you want to proceed once the tests pass? Shall we merge this, then later update the submodule URL once we merge in the package template changes?

astrofrog · 2014-05-16T21:37:02Z

@embray - thanks for all your work on this!

embray · 2014-05-16T22:02:43Z

Shall we merge this, then later update the submodule URL once we merge in the package template changes?

this

embray · 2014-05-16T22:45:53Z

🎊It passed!🎊

embray · 2014-05-16T22:46:31Z

Took for bloody ever too. Travis gets busy on a Friday afternoon 😦

Initial implementation of astropy_helpers

astrofrog · 2014-05-16T22:50:14Z

Awesome! 🎆 - ironic that the tests themselves are so fast but it takes several hours to run them.

Adding local python3 inv fix

switch to checking decoded version for build success message

Initial implementation of astropy_helpers

embray added 13 commits December 6, 2013 17:18

Removing some ignores that are not relevant to astropy_helpers

409a5ca

Adding the most recent ez_setup.py and the setuptools_boostrap script…

f277169

… from Astropy

Start an astropy_helpers package and initialize it wieht the setup_he…

2a5033a

…lpers and version_helpers modules from astropy; these are still heavily reliant on Astropy to work

Add a stub of a setup.py and a MANIFEST.in

6e937ea

Restore missing imports

2b32228

Updated this error string; though this function will likely be remove…

f4cfc39

…d at some point.

Removing the last vestiges of the astropy config system for now, as i…

e0fbe59

…t won't work with astropy_helpers in its current state. This can, and should be revisited once we have a better idea of what we're going to do about configuration.

Since setuptools and PyPI 'canonicalize' underscores to dashes might …

153a29e

…as well just used the dashed name explcitly

embray mentioned this pull request Dec 16, 2013

Create a separate astropy-helpers package to contain common infrastructure code for the core and affiliated packages astropy/astropy#1563

Merged

embray reviewed Dec 16, 2013
View reviewed changes

Should be version_info

8b8e74b

astrofrog reviewed Jan 13, 2014
View reviewed changes

embray added 8 commits January 13, 2014 17:49

Fix typo

1a37056

Only output an error warning if git submodule had a non-zero exit sta…

22c624b

…tus. It appears that in some versions of git even a successful run can output some text to stderr; see astropy/astropy#1563 (comment)

Combine setuptools_bootstrap into ah_bootstrap so as to reduce the nu…

1a75b26

…mber of needed 'bootstrap' scripts :)

Fix a bug in get_git_devstr--I think the old default functionality of…

c3f841b

… using __file__ may have made sense when this was used primarily by Astropy itself, but in the context of an affiliated package it makes no sense.

added astropy_helpers branch of package-template as a submodule in as…

a146998

…tropy_helpers.tests--this will allow us to use the package template itself as our test project for testing astropy_helpers :)

Add our first test of astropy_helpers--this test actually helped catc…

6cc226c

…h a bug which was fixed in c3f841b

Run setup.py from within the same python interpreter using setuptools…

21db0cb

…' sandboxing--this way we can capture test coverage properly

Enhancing this warning message in the hopes that it will also illumin…

1fb2ae8

…ate the test failure on Travis

Use git clone instead of shutil.copy--hopefully this will work better…

15b63aa

… to ensure the copy of package_template remains a valid git repo

embray added 2 commits May 16, 2014 17:13

Add Sphinx to the tox build deps so that the sphinx tests are run. Ad…

ecb2905

…d Python3.4 as well, and support for passing extra positional arguments to py.test

Some tweaks to get the sphinx exts (nominally) working on Python 3--h…

6e14183

…aven't tested building actual docs with them yet.

embray added a commit that referenced this pull request May 16, 2014

Merge pull request #1 from embray/master

dda4407

Initial implementation of astropy_helpers

embray merged commit dda4407 into astropy:master May 16, 2014

astrofrog pushed a commit that referenced this pull request Jun 2, 2014

Merge pull request #1 from astrofrog/adding_local_python3_inv-fix

1b0d353

Adding local python3 inv fix

eteq pushed a commit that referenced this pull request Apr 15, 2016

Merge pull request #1 from eteq/fix-cadair-py3err

237d284

switch to checking decoded version for build success message

astrofrog modified the milestone: 0.4.0 Jun 2, 2016

astrofrog pushed a commit to astropy/astropy-sphinx-theme that referenced this pull request Jan 20, 2018

Merge pull request astropy/astropy-helpers#1 from embray/master

818abb6

Initial implementation of astropy_helpers

astrofrog pushed a commit to astropy/astropy-sphinx-theme that referenced this pull request Jan 24, 2018

Merge pull request astropy/astropy-helpers#1 from embray/master

783230d

Initial implementation of astropy_helpers

astrofrog pushed a commit to astropy/sphinx-automodapi that referenced this pull request Jan 27, 2018

Merge pull request astropy/astropy-helpers#1 from embray/master

94840a7

Initial implementation of astropy_helpers

astrofrog pushed a commit to astropy/sphinx-automodapi that referenced this pull request Jan 27, 2018

Merge pull request astropy/astropy-helpers#1 from embray/master

fbfcbf5

Initial implementation of astropy_helpers

astrofrog pushed a commit to astropy/sphinx-astropy that referenced this pull request Jan 29, 2018

Merge pull request astropy/astropy-helpers#1 from embray/master

f269190

Initial implementation of astropy_helpers

javierggt mentioned this pull request Nov 12, 2020

Update sot/ska-sphinx-theme to Release 1.3.0 sot/skare3#525

Closed

javierggt mentioned this pull request Dec 2, 2020

Update sot/ska-sphinx-theme to Release 1.3.0 sot/skare3#544

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial implementation of astropy_helpers #1

Initial implementation of astropy_helpers #1

embray commented Dec 16, 2013

embray Dec 16, 2013

astrofrog Jan 12, 2014

astrofrog commented Dec 17, 2013

astrofrog commented Dec 17, 2013

embray commented Dec 17, 2013

rhewett commented Dec 17, 2013

astrofrog Jan 13, 2014

embray Jan 13, 2014

astrofrog commented May 16, 2014

astrofrog commented May 16, 2014

astrofrog commented May 16, 2014

embray commented May 16, 2014

embray commented May 16, 2014

embray commented May 16, 2014

embray commented May 16, 2014

astrofrog commented May 16, 2014

astrofrog commented May 16, 2014

embray commented May 16, 2014

embray commented May 16, 2014

embray commented May 16, 2014

astrofrog commented May 16, 2014

		@@ -0,0 +1,1558 @@
		# Licensed under a 3-clause BSD style license - see LICENSE.rst

Initial implementation of astropy_helpers #1

Initial implementation of astropy_helpers #1

Conversation

embray commented Dec 16, 2013

embray Dec 16, 2013

Choose a reason for hiding this comment

astrofrog Jan 12, 2014

Choose a reason for hiding this comment

astrofrog commented Dec 17, 2013

astrofrog commented Dec 17, 2013

embray commented Dec 17, 2013

rhewett commented Dec 17, 2013

astrofrog Jan 13, 2014

Choose a reason for hiding this comment

embray Jan 13, 2014

Choose a reason for hiding this comment

astrofrog commented May 16, 2014

astrofrog commented May 16, 2014

astrofrog commented May 16, 2014

embray commented May 16, 2014

embray commented May 16, 2014

embray commented May 16, 2014

embray commented May 16, 2014

astrofrog commented May 16, 2014

astrofrog commented May 16, 2014

embray commented May 16, 2014

embray commented May 16, 2014

embray commented May 16, 2014

astrofrog commented May 16, 2014