Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

merge scipy19 docs #3557

Merged
merged 15 commits into from
Nov 22, 2019
Merged

merge scipy19 docs #3557

merged 15 commits into from
Nov 22, 2019

Conversation

keewis
Copy link
Collaborator

@keewis keewis commented Nov 20, 2019 •
edited
Loading

This PR tries to merge the scipy19-docs branch into master while also making sure that the readthedocs build does not fail (see #3199 and #3369).

  • Passes black . && mypy . && flake8

There is also a preview page, but it currently fails because conda is taking too much memory (which the official builds do not). Does anyone know why that is?

@keewis keewis mentioned this pull request Nov 20, 2019
Merged
1 task
@keewis
Copy link
Collaborator Author

keewis commented Nov 20, 2019

something that could in the long run reduce the memory usage might be conda-metachannel (blog post), but I think the integration into conda is in its early stages?

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

it looks like it would be easier to do this by rebasing / cherry-picking onto master: there would not be as many conflicts as using merge would have (look at the list: 146 changed files). I'll cherry-pick and force push afterwards.

@dcherian
Copy link
Contributor

👍

max-sixty
max-sixty reviewed Nov 21, 2019
View reviewed changes
.pre-commit-config.yaml Outdated Show resolved Hide resolved
@rabernat
Copy link
Contributor

Sorry for creating this mess! 🤦‍♂️ Thank you for trying to clean it up.

rabernat and others added 5 commits November 21, 2019 15:11
@rabernat @keewis
Switch doc examples to use nbsphinx (pydata#3105)
e574663 
* switching out examples to use nbsphinx

* added jupyter_client to doc env

* added ipykernel to doc env
@rabernat @keewis
Replace sphinx_gallery with notebook (pydata#3106)
789a3d2 
* switching out examples to use nbsphinx

* added jupyter_client to doc env

* moved gallery to notebook
@rabernat @keewis
Allow other tutorial filename extensions (pydata#3121)
751ec0f 
* switching out examples to use nbsphinx

* added jupyter_client to doc env

* allow non netcdf tutorial files
@hetland @keewis
Added ROMS ocean model example notebook (pydata#3116)
1ab4652 
* change name of test env to xarray-tests (pydata#3110)

* ROMS_ocean_model example added

* Allow other tutorial filename extensions (pydata#3121)

* switching out examples to use nbsphinx

* added jupyter_client to doc env

* allow non netcdf tutorial files

* Changed load to xr.tutorial.open_dataset(), and
added some extra documentation.

* change name of test env to xarray-tests (pydata#3110)

* ROMS_ocean_model example added

* Changed load to xr.tutorial.open_dataset(), and
added some extra documentation.

* fixed colormap issues leftover from cmocean import

* Added intro paragraph to ROMS example notebook, removed comments, and added citation in whats-new.
@zbruick @keewis
Add an example of ERA5 and GRIB data & visualization to the gallery (p…
3f85659 
…ydata#3199)

* Adds an example of ERA5 and GRIB data to the gallery

* Add markdown narrative cells to GRIB example

* Update load method to use xr.tutorial

* Fix load method
@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

okay, now the cherry-picking is done and the number of touched files is down to a more reasonable 14, though I dropped the non-PR commits. I'm not sure which of these was really necessary, but I guess that's something that we will notice once we try to get CI and readthedocs to work?

The original branch is still available for comparison on scipy19-docs

@dcherian (or anyone else who knows a bit more about RTD than me), can you help me fix the readthedocs builds? I can't test anything as long as the environment creation fails.

@dcherian
Copy link
Contributor

Does anyhting on this page help? It's likely we have requested extra resources for xarray.

https://docs.readthedocs.io/en/stable/guides/build-using-too-many-resources.html#use-pip-when-possible

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

and now I ran into the shadowing issue we had in #3199. I think the culprit is cfgrib which uses extras_require to pull in xarray. Is there any way to tell conda not to use that? We could also try to pull it using pip, since that works (at least for me) and bypasses the download of the packaged xarray.

@dcherian
Copy link
Contributor

maybe pip is the solution to all of these problems

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

probably, but the only issue I found is this: https://pypi.org/project/iris/, which means iris is only available on conda-forge?

@dcherian
Copy link
Contributor

How about we pip install iris master just to see if we can get things to work?

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019 •
edited
Loading

I'm currently trying to move everything except python and iris into the pip section, let's see what kind of memory usage that gives us.

Edit: that almost worked, it failed in the execution state. I'll try to install iris from github.

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

iris from github would work, but cartopy has no wheels, so installing using pip fails. I'll stop investigating this for now and try to make CI pass.

@dcherian
Copy link
Contributor

image

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

now that CI passes we only need to get the readthedocs build to work (and maybe re-apply #3464?).

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

oh, and we have some warnings regarding missing document labels. Is there a way to reference labels in notebooks?

@keewis
update the documentation links
d502ae2 
Missing: section links in visualization_gallery.ipynb don't work yet,
also the one in io.rst (it has a unicode char).
@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

oh, and is there a reason why we merge into stable instead of master?

@dcherian
Copy link
Contributor

I just switched it so we can tweet about binderable examples ;)

@dcherian
Copy link
Contributor

Parsing rasterio geocoordinates? 🤷‍♂️

Is that the last thing to fix?

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

I would think so, but I think we can fix anything that pops up later in new PRs.

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

one thing that might be done in a future PR is enabling the HTML reprs?

@max-sixty
Copy link
Collaborator

Feel free to ignore since it's not a big deal relative to the benefits, but looks like this adds some recent changes on master to stable (e.g. https://github.com/pydata/xarray/pull/3557/files#diff-99289912975ea19afc02f51c240ca451R18)

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

yeah, should we merge to stable we should probably rebase first

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

@dcherian, I'd merge into master and then cherry-pick into stable?

@max-sixty
Copy link
Collaborator

Should we change the base of this PR to master? Then we can choose when we push to stable? (But let's not get stuck on this, fine to just go ahead if we can't resolve)

@dcherian dcherian changed the base branch from stable to master November 21, 2019 22:59
@dcherian
Copy link
Contributor

OK I jsut got too excited.

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

ah, maybe we should move the whats-new.rst entries?

@dcherian
Copy link
Contributor

Yes! there seems to be one new example entry

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

would that be documentation or enhancements?

@dcherian
Copy link
Contributor

Documentation

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019 •
edited
Loading

while we have an entry for #3116 the one for #3199 is missing (also, @rabernat, do you want an entry for switching to nbsphinx?)

@dcherian
Copy link
Contributor

Yes let's add appropriate entries for all those.

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

how about

- Switch doc examples to use nbsphinx and replace sphinx_gallery with notebook.
  (:pull:`3105`, :pull:`3106`, :pull:`3121`)
  By `Ryan Abernathey <https://github.com/rabernat>`
- Added example notebook demonstrating the visualization of ERA5 GRIB data.
  (:pull:`3199`).
  By `Zach Bruick <https://github.com/zbruick>` and `Stephan Siemen <https://github.com/StephanSiemen>`

@dcherian
Copy link
Contributor

👍

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

done

@keewis
Copy link
Collaborator Author

keewis commented Nov 21, 2019

I think we can merge after making sure RTD still builds?

@dcherian
Copy link
Contributor

Great work as usual @keewis !

@dcherian dcherian merged commit b0064b2 into pydata:master Nov 22, 2019
@keewis keewis mentioned this pull request Nov 24, 2019
Closed
@keewis keewis deleted the scipy19-docs branch November 25, 2019 11:29
dcherian added a commit to dcherian/xarray that referenced this pull request Nov 25, 2019
@dcherian
Merge remote-tracking branch 'upstream/master' into fix/plot-broadcast
794939e 
* upstream/master:
  Add drop to api.rst under pending deprecations (pydata#3561)
  replace duplicate method _from_vars_and_coord_names (pydata#3565)
  propagate indexes in to_dataset, from_dataset (pydata#3519)
  Switch examples to notebooks + scipy19 docs improvements (pydata#3557)
  fix whats-new.rst (pydata#3554)
dcherian added a commit to dcherian/xarray that referenced this pull request Dec 4, 2019
@dcherian
Merge remote-tracking branch 'upstream/master' into coarsen-reduce
d592623 
* upstream/master: (22 commits)
  Resolve the version issues on RTD (pydata#3589)
  Add bottleneck & rasterio git tip to upstream-dev CI (pydata#3585)
  update whats-new.rst (pydata#3581)
  Examples for quantile (pydata#3576)
  add cftime intersphinx entries (pydata#3577)
  Add pyXpcm to Related Projects doc page (pydata#3578)
  Reimplement quantile with apply_ufunc (pydata#3559)
  add environment file for binderized examples (pydata#3568)
  Add drop to api.rst under pending deprecations (pydata#3561)
  replace duplicate method _from_vars_and_coord_names (pydata#3565)
  propagate indexes in to_dataset, from_dataset (pydata#3519)
  Switch examples to notebooks + scipy19 docs improvements (pydata#3557)
  fix whats-new.rst (pydata#3554)
  Tweaks to release instructions (pydata#3555)
  Clarify conda environments for new contributors (pydata#3551)
  Revert to dev version
  0.14.1 whatsnew (pydata#3547)
  sparse option to reindex and unstack (pydata#3542)
  Silence sphinx warnings (pydata#3516)
  Numpy 1.18 support (pydata#3537)
  ...
dcherian added a commit to dcherian/xarray that referenced this pull request Dec 5, 2019
@dcherian
Merge remote-tracking branch 'upstream/master' into fix/user-coordinates
77c7c0f 
* upstream/master: (35 commits)
  fix plotting with transposed nondim coords. (pydata#3441)
  make coarsen reductions consistent with reductions on other classes (pydata#3500)
  Resolve the version issues on RTD (pydata#3589)
  Add bottleneck & rasterio git tip to upstream-dev CI (pydata#3585)
  update whats-new.rst (pydata#3581)
  Examples for quantile (pydata#3576)
  add cftime intersphinx entries (pydata#3577)
  Add pyXpcm to Related Projects doc page (pydata#3578)
  Reimplement quantile with apply_ufunc (pydata#3559)
  add environment file for binderized examples (pydata#3568)
  Add drop to api.rst under pending deprecations (pydata#3561)
  replace duplicate method _from_vars_and_coord_names (pydata#3565)
  propagate indexes in to_dataset, from_dataset (pydata#3519)
  Switch examples to notebooks + scipy19 docs improvements (pydata#3557)
  fix whats-new.rst (pydata#3554)
  Tweaks to release instructions (pydata#3555)
  Clarify conda environments for new contributors (pydata#3551)
  Revert to dev version
  0.14.1 whatsnew (pydata#3547)
  sparse option to reindex and unstack (pydata#3542)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@max-sixty max-sixty max-sixty left review comments

@rabernat rabernat rabernat approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@keewis @dcherian @rabernat @max-sixty @hetland @zbruick @mickaellalande