-
Notifications
You must be signed in to change notification settings - Fork 28.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[MINOR][DOC] Fix comments of ConvertToLocalRelation rule #23273
Conversation
Test build #99909 has finished for PR 23273 at commit
|
retest this please. |
Test build #99944 has finished for PR 23273 at commit
|
* | ||
* This is relatively simple as it currently handles only 2 single case: Project and Limit. | ||
* This rule currently handles 3 cases: [[Project]], [[Limit]] and [[Filter]]. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we should just not specify this in the docs. Are you sure the [[...]]
links work when you generate scaladoc? the test builder doesn't check. I've seen many errors/warnings generated by some types of links.
If so again maybe easier to restrict this to merely removing this line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@srowen Sorry, I found that the links just do not work for scaladoc, though they works in IDE like Intellij IDEA. I should have generated the docs to see if there's any problem.
I have changed [[...]]
to backticks in a new commit.
Also, I have some findings:
- Because
org/apache/spark/sql/catalyst
is excluded for doc generation, I include catalyst inSparkBuild.scala
and runbuild/sbt unidoc
, there are many errors/warnings, should we fix these problems? Seems simple but many places. - As for
[[...]]
, fully qualified class name should be used if a link points to a class in another package. If the link points to a class in the same package, package prefix could be omitted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
catalyst
isn't meant to be a public API (for our convenience, it's not all marked private in the source). That's why it's not in the docs and yeah that's why writing references to it won't work, as I recall.
For this comment I think it's simplest to remove it, or leave it but don't add references, or move it to an inline comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot @srowen for your explanation. Since it's not a public API and the code is clear, it makes sense to remove this line, this can also eliminate the needs to modify this line when new cases are added. I have removed this line in the new commit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also wouldn't tag this as SPARK-25212 as that was merged into 2.4.0. This is just a "MINOR" change
Test build #100006 has finished for PR 23273 at commit
|
retest this please. |
Test build #100032 has finished for PR 23273 at commit
|
retest this please. |
Test build #100054 has finished for PR 23273 at commit
|
@srowen The build failed twice because of the same reason. It seems we ran into SparkR CRAN feasibility check server problem, which blocks PR build. Error messages from log:
|
retest this please. |
Test build #100075 has finished for PR 23273 at commit
|
retest this please. |
Test build #100082 has finished for PR 23273 at commit
|
Merged to master |
Thank you! @srowen |
## What changes were proposed in this pull request? There are some comments issues left when `ConvertToLocalRelation` rule was added (see apache#22205/[SPARK-25212](https://issues.apache.org/jira/browse/SPARK-25212)). This PR fixes those comments issues. ## How was this patch tested? N/A Closes apache#23273 from seancxmao/ConvertToLocalRelation-doc. Authored-by: seancxmao <seancxmao@gmail.com> Signed-off-by: Sean Owen <sean.owen@databricks.com>
## What changes were proposed in this pull request? There are some comments issues left when `ConvertToLocalRelation` rule was added (see apache#22205/[SPARK-25212](https://issues.apache.org/jira/browse/SPARK-25212)). This PR fixes those comments issues. ## How was this patch tested? N/A Closes apache#23273 from seancxmao/ConvertToLocalRelation-doc. Authored-by: seancxmao <seancxmao@gmail.com> Signed-off-by: Sean Owen <sean.owen@databricks.com>
What changes were proposed in this pull request?
There are some comments issues left when
ConvertToLocalRelation
rule was added (see #22205/SPARK-25212). This PR fixes those comments issues.How was this patch tested?
N/A