Ginkgo version 1.0.0 release

[tcojean]

This issue is here to track the creation of the first Ginkgo release, which could be version 1.0.0 if we can get several CUDA conversions.

I created a milestone to track the release process, I added every issue and PR that we want for the release to this milestone. If you want something in particular to be part of this release, add an issue and add it to the release milestone. Currently there is:

• All open PRs
• all issues tagged with "Bug" except for two PRs (Implement atomic operations for custom value types #65 and Windows support #281).
• The issues related to the open PRs

TODO

• Create a release milestone and issue.
• Create a wiki help page containing the important steps for releasing Ginkgo.
• Create PRs to fix the open bugs which can be fixed, at least partially.
  • Fix Build fails with both SET_CUDA_HOST_COMPILER and CMAKE_CUDA_HOST_COMPILER enabled #71
  • Fix Remove circular linking dependencies in Ginkgo #203
  • Fix Wrong memory access in automatic strategy #130
  • Fix Check memory access quality of the CUDA executor #152
  • Fix Segmentation fault in CUDA Executor test #241
  • Fix fails to build on macOS with disabled cuda #270
  • Fix Reducing the amount of warnings #174
  • Fix Improvements to Doxygen #76
  • Fix CMake error: Unable to specify include directories for imported targets #234
  • Fix stopping_status is not covered by unit tests #242
  • Fix Problems related to mismatching executors #110
  • Fix Getting "fatal: not a git repository" during configure #240
  • Fix Better support for Mac and other shell versions #272
  • Fix Fix Issues detected by Valgrind #273
  • Fix Provide tests or implementation for some applies to multiple RHS #253
• Add a known issues page which contains the open bugs we cannot fix right now.
• Update the .gitlab-ci.yml file so that the CI system generates the documentation link: https://ginkgo-project.github.io/ginkgo/doc/1.0.0
• Compile a list of functionalities present in the future release.
• Update the external library's commit version.
• Tweak the compiled kernel sizes.
• Merge all open pull requests
• Ensure the milestone reached completion.
• Create a release/1.0.0 branch
• Update the version number everywhere appropriate. Double check any link.
• Run the CI pipelines and ensure there are no unexpected problems.
• Update any explicit reference to develop in the Readme or the CMakeLists.txt to master.
• Create a PR for merging the release branch into master.
• Add a tag to master to the new release.
• Compile both release and debug linux binaries with CPack, ensure they pass all tests and are complete.
• Use Github's release system to create a new release with the previously created tag. Add the release notes (functionalities present in this release) and compiled binaries.
• Merge back from master to develop. Do not select the changes replacing develop to master.
• Update the website to point on the master branch's documentation instead of develop?
• Update the website to advertise the release?

[gflegar]

Some notes:

• what do we do with the OMP kernels? AFAIK, trying to use the block-Jacobi preconditioner on OMP executor will result in the NotImplemented exception. Are we allowing this exception in the release version, or should we have a policy that such exception are not allowed?
• somewhat related is Full support for custom datatypes #53: trying to compile Ginkgo with a custom datatype will result in NotImplemented for some operations - the full list is in the issue (may not be updated). Do we consider this a problem?
• you may want to check out the description of the Gitflow workflow, especially the release branches part, for a description how the release should be handled. In particular, there should be no need to "freeze" Ginkgo, as the release dos not take over the develop branch.
• what do we do with the doxygen documentation? It's not very useful the way it looks now - it needs some introduction, and some sort of organized TOC by topics - what we currently have is just an index of various parts. Some parts are also missing, as the current setup only shows documented namespaces, classes, and their members. Since none of the namespaces have any documentation, there's no docs generated for any of the standalone functions (such as read, write, clone, lend, give, share, etc.).
• just a technical note: even though the master branch can be fast-forwarded to the final release commit, I would still force git to generate a new merge commit, just to have a better visual indication that this is a release when looking at Ginkgo's tree.
• another thing for the list: make sure that the CI system generates the docs on the correct link (that would probably be https://ginkgo-project.github.io/ginkgo/doc/1.0.0)

[tcojean]

Note on documentation, I did hesitate to add #76 to this milestone... It should not be too long to fix but is much needed.

[tcojean]

Thanks @gflegar for the comments. The gitflow is helpful for managing the release efforts in terms of commits. I will add this to the wiki page I am currently writing. I think the release process itself can be quite fast if we have a correct setup for testing and doing the packaging (let's say a day at most), so I think merging the fixes directly to develop and agreeing on not merging any new PR until we are done with actually pushing the release to the world should be enough. What do you think?

I also want to mention that both Github and Gitlab facilitate the release process. Here are the respective pages: Github release page and Gitlab release page.

[tcojean]

• On OpenMP kernels, it depends what we aim to support as part of this release. Either we say in the release notes we have a full OpenMP implementation but its quality is not guaranteed, in which case we have to implement all NotImplemented kernels, or we explicitly note what is not there (both as a release note and I guess a known issue in that case).
• On Full support for custom datatypes #53, I would have a similar approach. Either we say we have full support for custom value types, or we say we have backend support but not all kernels and formats allow its use yet (the user could still use it as part of its own class).

I want to mention that the task Compile a list of functionalities present in the future release. is actually a very important (and hard, and long) one. Maybe we should split it into several subtasks.

I agree on the having an explicit merge commit in the master branch and the documentation issue. I will note this down.

[tcojean]

#174 could also be a good candidate for the release. It's warnings so it's fine if they stay, but better if they are gone.

[tcojean]

I created a wiki page with the instructions I could think of for the release process. Feel free to contribute. There may be multiple typos...

[tcojean]

@ginkgo-project/developers I would like to collect everyone's thoughts on the following issues we rose with @gflegar.

• Do we implement all OpenMP kernels (for example the Block Jacobi), see Goran's previous comment and my answer.
• Do we implement Full support for custom datatypes #53 (again, check Goran's comment)
• Do we improve documentation through Improvements to Doxygen #76 and other issues brought up in Goran's comment?
• Do we maybe fix Reducing the amount of warnings #174 ?

[hartwiganzt]

On the OMP issue: there are in total about 25 NOT_IMPLEMENTED in the omp branch. I would say: let's copy complete those - in the worst case with the reference implementation and some note saying that this still needs to be parallelized.

[tcojean]

Note from the discussions, we're not doing #53 as it seems like it would take too much time. But we will try to fix all the other problems listed here.

[gflegar]

I guess we are delaying this? The deadline is tomorrow, and there are quite a few open issues in this milestone.

[tcojean]

Yes, the deadline was a loose tentative one. Looks like we will have to postpone it, hopefully not by much.

[yhmtsai]

Do we implement all spmv for all matrix formats on all executors?
Maybe use for loop to make it work first?

[hartwiganzt]

I don't think that makes too much sense, having all formats for all executors. I feel: we obviously need the reference executor for all formats, and if there is a format selected that is not supported by the executor, we fall back to the reference executor. Is that a reasonable path forward?

[yhmtsai]

I mean the operation apply to multiple vectors of coo, csr, dense, ell hybrid and sellp on all executor.
The following are the status of formats

1. have implementations and tests
   • coo, csr, dense, ell, hybrid, sellp on reference executor
   • coo, ell, sellp on cuda executor
   • dense on omp executor
2. have implementations but don't have tests
   • hybrid and dense on cuda executor
   • coo, csr, ell, sellp on omp executor
3. don't have implementations
   • csr on cuda executor

[hartwiganzt]

I now understand - sorry.
I guess it would be good to close the gaps... for now, a loop might be acceptable.

[tcojean]

Thanks for the notification @yhmtsai and for already fixing a part of that. I put this in a new issue (#253) to track this separately.