Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-3615][Streaming]Fix Kafka unit test hard coded Zookeeper port issue #2483

Closed
wants to merge 1 commit into from
Closed

[SPARK-3615][Streaming]Fix Kafka unit test hard coded Zookeeper port issue #2483

wants to merge 1 commit into from

Conversation

jerryshao
Copy link
Contributor

Details can be seen in SPARK-3615.

@jerryshao jerryshao changed the title Fix Kafka unit test hard coded Zookeeper port issue [SPARK-3615][Streaming]Fix Kafka unit test hard coded Zookeeper port issue Sep 22, 2014
@jerryshao
Copy link
Contributor Author

@pwendell @tdas , mind taking a look at this? Thanks a lot.

@SparkQA
Copy link

SparkQA commented Sep 22, 2014

QA tests have started for PR 2483 at commit 8555563.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 22, 2014

QA tests have finished for PR 2483 at commit 8555563.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

tdas
tdas reviewed Sep 23, 2014
View reviewed changes
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
try {
val brokerProps = getBrokerConfig(brokerPort, s"$zkHost:$zkPort")
brokerConf = new KafkaConfig(brokerProps)
server = new KafkaServer(brokerConf)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there no way to specify the KafkaServer port as 0, and get back the actual port it bound to?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi TD, thanks for reviewing. Actually there's no way for KafkaServer to read back the port it bound to, so I wrote like this to try different ports.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alright. just one more round of testing, and will merge it if it passes.

@tdas
Copy link
Contributor

tdas commented Sep 23, 2014

LGTM, except one minor confirmation. Also, lets test this a number of times to see its flakiness.

@tdas
Copy link
Contributor

tdas commented Sep 23, 2014

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Sep 23, 2014

QA tests have started for PR 2483 at commit 8555563.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 23, 2014

QA tests have finished for PR 2483 at commit 8555563.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 23, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20717/

@tdas
Copy link
Contributor

tdas commented Sep 23, 2014

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Sep 23, 2014

QA tests have started for PR 2483 at commit 8555563.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 23, 2014

QA tests have finished for PR 2483 at commit 8555563.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 23, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20725/

@tdas
Copy link
Contributor

tdas commented Sep 24, 2014

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Sep 24, 2014

QA tests have started for PR 2483 at commit 8555563.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 24, 2014

QA tests have finished for PR 2483 at commit 8555563.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 24, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20743/

@asfgit asfgit closed this in 74fb2ec Sep 25, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jerryshao @SparkQA @tdas