Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-4508] [SQL] build native date type to conform behavior to Hive #4325

Closed
wants to merge 8 commits into from
Closed

[SPARK-4508] [SQL] build native date type to conform behavior to Hive #4325

wants to merge 8 commits into from

Conversation

adrian-wang
Copy link
Contributor

The previous #3732 is reverted due to some test failure.
Have fixed that.

@SparkQA
Copy link

SparkQA commented Feb 3, 2015

Test build #26617 has started for PR 4325 at commit 096e20d.

  • This patch merges cleanly.

@marmbrus
Copy link
Contributor

marmbrus commented Feb 3, 2015

Thanks for the quick fix.

LGTM pending tests.

@SparkQA
Copy link

SparkQA commented Feb 3, 2015

Test build #26617 has finished for PR 4325 at commit 096e20d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26617/
Test PASSed.

cloud-fan
cloud-fan reviewed Feb 3, 2015
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/types/DateUtils.scala
/**
* helper function to convert between Int value of days since 1970-01-01 and java.sql.Date
*/
object DateUtils {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mark it private[spark]?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could be useful even outside of spark.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's not necessary, just do not make it public, or we can not change it anymore.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code gen need that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That make sense, thanks!

@SparkQA
Copy link

SparkQA commented Feb 3, 2015

Test build #26639 has started for PR 4325 at commit e46735c.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Feb 3, 2015

Test build #26639 has finished for PR 4325 at commit e46735c.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26639/
Test FAILed.

@adrian-wang
Copy link
Contributor Author

retest this please.

@SparkQA
Copy link

SparkQA commented Feb 3, 2015

Test build #26642 has started for PR 4325 at commit 096e20d.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Feb 3, 2015

Test build #26642 has finished for PR 4325 at commit 096e20d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26642/
Test PASSed.

@asfgit asfgit closed this in db821ed Feb 3, 2015
asfgit pushed a commit that referenced this pull request Feb 3, 2015
@adrian-wang @marmbrus
[SPARK-4508] [SQL] build native date type to conform behavior to Hive
6e244cf 
The previous #3732 is reverted due to some test failure.
Have fixed that.

Author: Daoyuan Wang <daoyuan.wang@intel.com>

Closes #4325 from adrian-wang/datenative and squashes the following commits:

096e20d [Daoyuan Wang] fix for mixed timezone
0ed0fdc [Daoyuan Wang] fix test data
a2fdd4e [Daoyuan Wang] getDate
c37832b [Daoyuan Wang] row to catalyst
f0005b1 [Daoyuan Wang] add date in sql parser and java type conversion
024c9a6 [Daoyuan Wang] clean some import order
d6715fc [Daoyuan Wang] refactoring Date as Primitive Int internally
374abd5 [Daoyuan Wang] spark native date type support

(cherry picked from commit db821ed)
Signed-off-by: Michael Armbrust <michael@databricks.com>
@viper-kun viper-kun mentioned this pull request Feb 4, 2015
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@adrian-wang @SparkQA @marmbrus @AmplabJenkins @davies @cloud-fan