Cleaned up docs. #1642

gunjanbaid · 2017-07-26T23:32:03Z

Cleaned up formatting, language, and broken links. @fnothaft Let me know if it's easier for me to open multiple PRs (one for each file?).

coveralls · 2017-07-26T23:44:19Z

Coverage remained the same at 83.961% when pulling 5f8d939 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

AmplabJenkins · 2017-07-26T23:53:22Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2288/
Test PASSed.

fnothaft · 2017-07-27T03:44:37Z

Thanks @gunjanbaid! A single PR is a-OK by me! I'll make a pass shortly.

fnothaft

Thanks @gunjanbaid! Looks solid!

I saw lots of places where we lost anchors/anchor links in the documentation. I'm guessing this was intentional? My preference is to keep the links, as they aid navigation in the HTML and PDF copies. What's your general line on this?

fnothaft · 2017-07-27T03:45:32Z

docs/source/01_intro.md

-* View our software artifacts on Maven Central at [http://search.maven.org/#search%7Cga%7C1%7Corg.bdgenomics](http://search.maven.org/#search%7Cga%7C1%7Corg.bdgenomics)
-* See our snapshots at [https://oss.sonatype.org/index.html#nexus-search;quick~bdgenomics](https://oss.sonatype.org/index.html#nexus-search;quick~bdgenomics)
-* Look at our CHANGES file at [https://github.com/bigdatagenomics/adam/blob/master/CHANGES.md](https://github.com/bigdatagenomics/adam/blob/master/CHANGES.md)
+* [Follow](https://twitter.com/bigdatagenomics/) our Twitter account


Thanks for updating this, it reads much nicer! Would you mind copying this over to README.md? There is a very similar section of links in there, IIRC.

fnothaft · 2017-07-27T03:46:29Z

docs/source/02_installation.md

@@ -1,4 +1,4 @@
-# Building ADAM from Source {#build-from-source}
+# Building ADAM from Source


OOC, why'd we lose the anchor tag here?

I didn't think this was the correct formatting for these anchors, based on adam docs. There seemed to be two issues:

The {#build-from-source} shows up on the page, which looks weird.

Links to such anchors don't work, for example http://adam.readthedocs.io/en/latest/source/02_installation/#python-build just goes to the top of the page.

Since you can link to headers with just the text of the header, separated by dashes (for example, http://adam.readthedocs.io/en/latest/source/02_installation/#running-adam), I just removed the anchors and updated existing links to this format.

The anchors in the first format don't work when displayed on Github but they do work after generating the docs. Is the second form also compatible with Sphinx?

For now, I would consider the output of the pandoc scripts (docs/build.sh) to be the golden copy.

OOC, did one of us set up the readthedocs page?

Yes, I set up the one here https://readthedocs.org/projects/adam/

Mine is gunjanbaid

Thanks! I am fnothaft, same as here.

ok, if you log into readthedocs, you should be able to go to https://readthedocs.org/dashboard/adam and hit the Admin button

Thanks @heuermh! Works for me now.

edit: My readthedocs username is devin.petersohn

fnothaft · 2017-07-27T03:47:03Z

docs/source/02_installation.md

@@ -49,15 +49,15 @@ to have the Spark binaries on your system; prebuilt binaries can be downloaded f
 https://amplab.cs.berkeley.edu/jenkins/job/ADAM/) builds ADAM against Spark versions 1.6.1 and 2.0.0,
 Scala versions 2.10 and 2.11, and Hadoop versions 2.3.0 and 2.6.0.

-Once this alias is in place, you can run ADAM by simply typing `adam-submit` at the commandline, e.g.
+Once this alias is in place, you can run ADAM by simply typing `adam-submit` at the commandline.


While we're at it, let's change commandline -> command line

fnothaft · 2017-07-27T03:47:46Z

docs/source/02_installation.md


 ```bash
 $ adam-submit
 ```

-## Building for Python {#python-build}
+## Building for Python


Why'd we lose the anchor tag here and the anchor link two lines down?

Explanation for anchor tags provided in earlier comment.

For the anchor link, I thought it was for an outdated anchor that had been removed, there's no #python anchor. When you click on that link on this page, it does not take you anywhere.

fnothaft · 2017-07-27T03:48:02Z

docs/source/02_installation.md

@@ -100,7 +100,7 @@ ASSEMBLY_JAR="$(ls -1 "$ASSEMBLY_DIR" | grep "^adam[0-9A-Za-z\.\_-]*\.jar$" | gr
 export PYSPARK_SUBMIT_ARGS="--jars ${ASSEMBLY_DIR}/${ASSEMBLY_JAR} --driver-class-path ${ASSEMBLY_DIR}/${ASSEMBLY_JAR} pyspark-shell"
 ```

-This assumes that the [ADAM JARs have already been built](#build-from-source).
+This assumes that the [ADAM JARs have already been built](#).


Ditto here RE: anchor link.

Clicking on previous link from docs page does not have any effect. Updated link takes you to the top header. I can update this to be #building-adam-from-source, that is probably more clear than just #.

fnothaft · 2017-07-27T03:51:03Z

docs/source/40_deploying_ADAM.md

-
-Note, the steps to register your ssh key and create the template boxes below
- need only be done once.
+Note, the steps to register your ssh key and create the template boxes below only need to be done once.


Thoughts on removing below in this line? Reads a bit awkward to me.

Sounds good, will update.

fnothaft · 2017-07-27T03:52:10Z

docs/source/40_deploying_ADAM.md

-in AWS console that your instances have terminated to avoid unintended costs.
-
+spot instances in order to save on costs. To avoid unintended costs, 
+it's a good idea to use the AWS console to double check that your 


Thoughts on removing all contractions in the docs? My preference is to not use contractions in formal writing.

I would be in favor of this! Will update.

fnothaft · 2017-07-27T03:52:36Z

docs/source/40_deploying_ADAM.md


 #### Access Spark GUI
+


While we're cleaning up this section, would the header read better as Accessing the Spark GUI?

Will update.

fnothaft · 2017-07-27T03:53:33Z

docs/source/40_deploying_ADAM.md

 properly configured for your YARN deployment. Typically, if your Spark
-assembly is configured properly to use YARN, there will be symbolic link at
+assembly is properly configured to use YARN, there will be symbolic link at


be symbolic -> be a symbolic

fnothaft · 2017-07-27T03:54:13Z

docs/source/40_deploying_ADAM.md

@@ -253,9 +254,9 @@ include:

 * [adam-kmers](https://github.com/BD2KGenomics/toil-scripts/tree/master/src/toil_scripts/adam_kmers):
  this workflow was demonstrated in [@vivian16] and sets up a Spark cluster
-  which then runs ADAM's [`countKmers` CLI](#countKmers).
+  which then runs ADAM's `countKmers` CLI.


Lost the anchor link.

Also two lines below.

gunjanbaid · 2017-07-27T04:37:50Z

docs/source/40_deploying_ADAM.md

 Shutdown the cluster using:
 ```
 cgcloud terminate-cluster -c cluster1 spark
 ```

 #### CGCoud options and Spot Instances
-View help docs for all options of the the `cgcloud create-cluster` command:
+
+View help docs for all options of the `cgcloud create-cluster` command:
 ```
 cgcloud create-cluster -h


Some code blocks with bash commands (like these) start with $, while some don't. Is there a preferred style? I can update so they are all consistent.

For me, if the bash code block shows one or commands that the user would be likely to copy and paste into a bash shell, then I typically don't include the $.

mvn install ./bin/adam-submit transformAlignments foo.bam bar.adam spark-submit conductor-0.5-SNAPSHOT-distribution.jar hdfs://bar.adam s3://bar.adam

If the code block is showing the output of a command, then I would include the $.

$ adam-submit --help Using ADAM_MAIN=org.bdgenomics.adam.cli.ADAMMain Using SPARK_SUBMIT=/usr/local/bin/spark-submit e 888~-_ e e e d8b 888 \ d8b d8b d8b /Y88b 888 | /Y88b d888bdY88b / Y88b 888 | / Y88b / Y88Y Y888b /____Y88b 888 / /____Y88b / YY Y888b / Y88b 888_-~ / Y88b / Y888b Usage: adam-submit [<spark-args> --] <adam-args> ...

coveralls · 2017-07-27T17:19:08Z

Coverage remained the same at 83.961% when pulling 3329607 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

AmplabJenkins · 2017-07-27T17:28:33Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2292/
Test PASSed.

coveralls · 2017-07-27T17:39:56Z

Coverage remained the same at 83.961% when pulling 7643f64 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

AmplabJenkins · 2017-07-27T17:49:22Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2293/
Test PASSed.

coveralls · 2017-07-27T18:00:52Z

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

AmplabJenkins · 2017-07-27T18:09:38Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2294/
Test PASSed.

coveralls · 2017-07-27T18:21:00Z

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

AmplabJenkins · 2017-07-27T18:29:54Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2295/
Test PASSed.

coveralls · 2017-07-27T18:41:24Z

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

AmplabJenkins · 2017-07-27T18:50:51Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2296/
Test PASSed.

coveralls · 2017-07-27T19:02:12Z

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

AmplabJenkins · 2017-07-27T19:11:11Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2297/
Test PASSed.

coveralls · 2017-07-27T19:22:22Z

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

AmplabJenkins · 2017-07-27T19:32:19Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2298/
Test PASSed.

fnothaft

LGTM! Thanks @gunjanbaid!

fnothaft · 2017-08-02T16:33:11Z

docs/source/40_deploying_ADAM.md


-From ADAM shell, or as parameter to ADAM submit, you would refer HDFS URLs
-such as:
+From the ADAM shell, or as a parameter to ADAM submit, you would refer to HDFS URLs like this:


Note to self: break this line at 80 characters when merging.

fnothaft · 2017-08-02T16:35:02Z

docs/source/40_deploying_ADAM.md

-on your local machine point your web-browser to http://ip_of_spark_master:4040/
+8080 on `spark-master`, go to Security Groups in the AWS console
+and open inbound TCP for those ports from your local IP address. Find the 
+IP address of `spark-master`, which is part of the Linux command prompt. On your local machine, 


Note to self: break this line at 80 characters when merging.

fnothaft · 2017-08-02T16:35:41Z

docs/source/40_deploying_ADAM.md

@@ -520,8 +521,8 @@ For those groups with access to a HPC cluster with [Slurm](https://en.wikipedia.
 managing a number of compute nodes with local and/or network attached storage, it is possible to spin up a 
 temporary Spark cluster for use by ADAM.

-While the full IO bandwidth benefits of Spark processing are likely best realized through a set of 
-co-located compute/storage nodes, depending on your network setup you may find Spark deployed on HPC 
+The full IO bandwidth benefits of Spark processing are likely best realized through a set of 


Note to self: break this line at 80 characters when merging.

fnothaft · 2017-08-02T16:35:52Z

docs/source/40_deploying_ADAM.md

-We'd love to hear feedback on your experience running ADAM on HPC/Slurm or other deployment architectures, 
-and let us know of any problems you run into via the mailing list or Gitter.
+
+We would love to hear feedback on your experience running ADAM on HPC/Slurm or other deployment architectures. 


Note to self: break this line at 80 characters when merging.

fnothaft · 2017-08-02T16:55:05Z

Thanks @gunjanbaid! I've merged this manually as 6573997 and 9032698.

gunjanbaid and others added 9 commits July 26, 2017 11:11

Added a section for homebrew.

3071243

Removed URL text from view.

c5bac1d

Minor edits to 01_intro.md.

7b98f1c

Removed broken links, unnecessary heading anchors.

1b2948c

Update 02_installation.md

5a14e51

Minor edits to 30_running_example.md.

caa5fd3

Minor edits to 40_deploying_ADAM.md.

51758a5

More minor edits.

136606f

More minor edits.

5f8d939

fnothaft requested changes Jul 27, 2017

View reviewed changes

gunjanbaid commented Jul 27, 2017

View reviewed changes

gunjanbaid added 3 commits July 27, 2017 10:06

Moved intro paragraph to top, consistent w/ README

1133eae

Updating links to be consistent w/ intro in docs

2b96553

Added anchors links back in.

3329607

gunjanbaid added 4 commits July 27, 2017 10:21

Added anchor links and tags back in.

c3b9778

Added anchor links and tags back in.

bddf45f

Removing extra hyphen.

94143ba

Removing extra line.

f34b381

gunjanbaid added 2 commits July 27, 2017 10:30

Adding minor changes based on reviewer comments.

baf6233

Removed $ from code cells with just commands.

7643f64

Removed contractions.

7cb8ea5

gunjanbaid mentioned this pull request Jul 27, 2017

Algorithms docs formatting #1645

Closed

fnothaft approved these changes Aug 2, 2017

View reviewed changes

heuermh approved these changes Aug 2, 2017

View reviewed changes

fnothaft closed this Aug 2, 2017

heuermh added this to the 0.23.0 milestone Dec 7, 2017

		@@ -1,4 +1,4 @@
		# Building ADAM from Source {#build-from-source}
		# Building ADAM from Source

Cleaned up docs. #1642

Cleaned up docs. #1642

Conversation

gunjanbaid commented Jul 26, 2017 • edited Loading

coveralls commented Jul 26, 2017 • edited Loading

AmplabJenkins commented Jul 26, 2017

fnothaft commented Jul 27, 2017

fnothaft left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devin-petersohn Jul 30, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Jul 27, 2017 • edited Loading

AmplabJenkins commented Jul 27, 2017

coveralls commented Jul 27, 2017 • edited Loading

AmplabJenkins commented Jul 27, 2017

coveralls commented Jul 27, 2017 • edited Loading

AmplabJenkins commented Jul 27, 2017

coveralls commented Jul 27, 2017 • edited Loading

AmplabJenkins commented Jul 27, 2017

coveralls commented Jul 27, 2017 • edited Loading

AmplabJenkins commented Jul 27, 2017

coveralls commented Jul 27, 2017 • edited Loading

AmplabJenkins commented Jul 27, 2017

coveralls commented Jul 27, 2017 • edited Loading

AmplabJenkins commented Jul 27, 2017

fnothaft left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fnothaft commented Aug 2, 2017

gunjanbaid commented Jul 26, 2017 •

edited

Loading

coveralls commented Jul 26, 2017 •

edited

Loading

devin-petersohn Jul 30, 2017 •

edited

Loading

coveralls commented Jul 27, 2017 •

edited

Loading

coveralls commented Jul 27, 2017 •

edited

Loading

coveralls commented Jul 27, 2017 •

edited

Loading

coveralls commented Jul 27, 2017 •

edited

Loading

coveralls commented Jul 27, 2017 •

edited

Loading

coveralls commented Jul 27, 2017 •

edited

Loading

coveralls commented Jul 27, 2017 •

edited

Loading