Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-4821] [mllib] [python] [docs] Fix for pyspark.mllib.rand doc #3669

Closed
wants to merge 3 commits into from

Conversation

jkbradley
Copy link
Member

  • small doc edit
  • include edit to make IntelliJ happy

CC: @davies @mengxr

Note to @davies -- this does not fix the "WARNING: Literal block expected; none found." warnings since that seems to involve spacing which IntelliJ does not like. (Those warnings occur when generating the Python docs.)

@jkbradley
Copy link
Member Author

CC: @pwendell This fixes a problem with missing documentation in the current build (and in branch-1.2).

@@ -36,10 +36,10 @@ pyspark.mllib.linalg module
:undoc-members:
:show-inheritance:

pyspark.mllib.random module
pyspark.mllib.rand module
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we still use random as the public module name, rand is only used internal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be fixed by

diff --git a/python/pyspark/mllib/__init__.py b/python/pyspark/mllib/__init__.py
index 5030a65..cba540a 100644
--- a/python/pyspark/mllib/__init__.py
+++ b/python/pyspark/mllib/__init__.py
@@ -32,7 +32,7 @@ import sys
 import rand as random
 random.__name__ = 'random'
 random.RandomRDDs.__module__ = __name__ + '.random'

+sys.modules[__name__ + '.random'] = random

 class RandomModuleHook(object):
     """

@jkbradley
Copy link
Member Author

@davies Thanks very much! Updated. I made a separate JIRA for fixing Python doc annotations:
[https://issues.apache.org/jira/browse/SPARK-4822]

@jkbradley
Copy link
Member Author

That last commit fixes a doc build warning:

.../spark/python/docs/pyspark.streaming.rst:2: WARNING: Title underline too short.

@SparkQA
Copy link

SparkQA commented Dec 11, 2014

Test build #24325 has finished for PR 3669 at commit c51eca4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 11, 2014

Test build #24331 has finished for PR 3669 at commit 4587868.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 11, 2014

Test build #24330 has finished for PR 3669 at commit 8cb073c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@jkbradley
Copy link
Member Author

Ping @mengxr @davies

@davies
Copy link
Contributor

davies commented Dec 16, 2014

LGTM.

asfgit pushed a commit that referenced this pull request Dec 17, 2014
+ small doc edit
+ include edit to make IntelliJ happy

CC: davies  mengxr

Note to davies  -- this does not fix the "WARNING: Literal block expected; none found." warnings since that seems to involve spacing which IntelliJ does not like.  (Those warnings occur when generating the Python docs.)

Author: Joseph K. Bradley <joseph@databricks.com>

Closes #3669 from jkbradley/python-warnings and squashes the following commits:

4587868 [Joseph K. Bradley] fixed warning
8cb073c [Joseph K. Bradley] Updated based on davies recommendation
c51eca4 [Joseph K. Bradley] Updated rst file for pyspark.mllib.rand doc.  Small doc edit.  Small include edit to make IntelliJ happy.

(cherry picked from commit affc3f4)
Signed-off-by: Xiangrui Meng <meng@databricks.com>
@asfgit asfgit closed this in affc3f4 Dec 17, 2014
@mengxr
Copy link
Contributor

mengxr commented Dec 17, 2014

Merged into master and branch-1.2. Thanks!

JoshRosen pushed a commit to JoshRosen/spark that referenced this pull request Jan 12, 2015
+ small doc edit
+ include edit to make IntelliJ happy

CC: davies  mengxr

Note to davies  -- this does not fix the "WARNING: Literal block expected; none found." warnings since that seems to involve spacing which IntelliJ does not like.  (Those warnings occur when generating the Python docs.)

Author: Joseph K. Bradley <joseph@databricks.com>

Closes apache#3669 from jkbradley/python-warnings and squashes the following commits:

4587868 [Joseph K. Bradley] fixed warning
8cb073c [Joseph K. Bradley] Updated based on davies recommendation
c51eca4 [Joseph K. Bradley] Updated rst file for pyspark.mllib.rand doc.  Small doc edit.  Small include edit to make IntelliJ happy.

Conflicts:
	python/docs/pyspark.streaming.rst
	python/pyspark/mllib/feature.py
asfgit pushed a commit that referenced this pull request Jan 13, 2015
…o branch-1.1

This backports #3216 and #3669 to `branch-1.1` in order to fix the PySpark unit tests.

Author: Joseph K. Bradley <joseph@databricks.com>
Author: Davies Liu <davies@databricks.com>

Closes #4011 from JoshRosen/pyspark-rand-fix-1.1-backport and squashes the following commits:

ace4cb6 [Joseph K. Bradley] [SPARK-4821] [mllib] [python] [docs] Fix for pyspark.mllib.rand doc
7ae5a1c [Davies Liu] [SPARK-4348] [PySpark] [MLlib] rename random.py to rand.py
@jkbradley jkbradley deleted the python-warnings branch July 25, 2016 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants