Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Intermittent HelixBootstrapUpgradeToolTest failure #1147

Closed
jsjtzyy opened this issue Apr 2, 2019 · 1 comment
Closed

Intermittent HelixBootstrapUpgradeToolTest failure #1147

jsjtzyy opened this issue Apr 2, 2019 · 1 comment
Assignees
@pnarayanan

Comments

@jsjtzyy
Copy link
Contributor

jsjtzyy commented Apr 2, 2019 

[2019-04-02 10:15:37,284] INFO Registered instance config change listeners for Helix manager at localhost:2201 (com.github.ambry.clustermap.HelixClusterManager)
[2019-04-02 10:15:37,285] INFO Received initial notification for live instance change from DC1 (com.github.ambry.clustermap.HelixClusterManager)
[2019-04-02 10:15:37,285] INFO Registered live instance change listeners for Helix manager at localhost:2201 (com.github.ambry.clustermap.HelixClusterManager)
[2019-04-02 10:15:37,291] INFO Dynamic addition of new nodes is not yet supported, ignoring InstanceConfig ZnRecord=yinzhang-ld1.linkedin.biz_18115, {HELIX_HOST=yinzhang-ld1.linkedin.biz, HELIX_PORT=18115, datacenter=DC0, rackId=null, schemaVersion=0, sslPort=28115, xid=64}{/mnt0={capacityInBytes=107374182400, diskState=AVAILABLE, Replicas=341:1073741824:defaultPartitionClass,352:1073741824:defaultPartitionClass,}, /mnt1={capacityInBytes=107374182400, diskState=AVAILABLE, Replicas=328:1073741824:defaultPartitionClass,344:1073741824:defaultPartitionClass,}, /mnt2={capacityInBytes=107374182400, diskState=AVAILABLE, Replicas=324:1073741824:defaultPartitionClass,359:1073741824:defaultPartitionClass,364:1073741824:defaultPartitionClass,}, /mnt3={capacityInBytes=107374182400, diskState=AVAILABLE, Replicas=320:1073741824:defaultPartitionClass,}, /mnt4={capacityInBytes=107374182400, diskState=AVAILABLE, Replicas=11:1073741825:specialPartitionClass,}, /mnt5={capacityInBytes=107374182400, diskState=AVAILABLE, Replicas=336:1073741824:defaultPartitionClass,350:1073741824:defaultPartitionClass,355:1073741824:defaultPartitionClass,365:1073741824:defaultPartitionClass,}}{SEALED=[], STOPPED=[]}, Stat=Stat {_version=0, _creationTime=0, _modifiedTime=0, _ephemeralOwner=0} (com.github.ambry.clustermap.HelixClusterManager)

java.io.IOException: Encountered startup blocking exception while parsing json, connecting or initializing in the local DC

	at com.github.ambry.clustermap.HelixClusterManager.<init>(HelixClusterManager.java:182)
	at com.github.ambry.clustermap.HelixClusterAgentsFactory.getClusterMap(HelixClusterAgentsFactory.java:55)
	at com.github.ambry.clustermap.HelixBootstrapUpgradeUtil.startClusterManager(HelixBootstrapUpgradeUtil.java:726)
	at com.github.ambry.clustermap.HelixBootstrapUpgradeUtil.updateClusterMapInHelix(HelixBootstrapUpgradeUtil.java:426)
	at com.github.ambry.clustermap.HelixBootstrapUpgradeUtil.bootstrapOrUpgrade(HelixBootstrapUpgradeUtil.java:176)
	at com.github.ambry.clustermap.HelixBootstrapUpgradeToolTest.writeBootstrapOrUpgrade(HelixBootstrapUpgradeToolTest.java:348)
	at com.github.ambry.clustermap.HelixBootstrapUpgradeToolTest.testEverything(HelixBootstrapUpgradeToolTest.java:332)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:24)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.IllegalStateException: Expected replica capacity 1073741825 is different from the capacity of an existing replica 1073741824
	at com.github.ambry.clustermap.HelixClusterManager$ClusterChangeHandler.ensurePartitionAbsenceOnNodeAndValidateCapacity(HelixClusterManager.java:682)
	at com.github.ambry.clustermap.HelixClusterManager$ClusterChangeHandler.initializeDisksAndReplicasOnNode(HelixClusterManager.java:647)
	at com.github.ambry.clustermap.HelixClusterManager$ClusterChangeHandler.initializeInstances(HelixClusterManager.java:487)
	at com.github.ambry.clustermap.HelixClusterManager$ClusterChangeHandler.onInstanceConfigChange(HelixClusterManager.java:450)
	at org.apache.helix.manager.zk.CallbackHandler.invoke(CallbackHandler.java:381)
	at org.apache.helix.manager.zk.CallbackHandler.init(CallbackHandler.java:599)
	at org.apache.helix.manager.zk.CallbackHandler.<init>(CallbackHandler.java:220)
	at org.apache.helix.manager.zk.ZKHelixManager.addListener(ZKHelixManager.java:358)
	at org.apache.helix.manager.zk.ZKHelixManager.addInstanceConfigChangeListener(ZKHelixManager.java:415)
	at com.github.ambry.clustermap.HelixClusterManager.lambda$new$0(HelixClusterManager.java:155)
	at java.lang.Thread.run(Thread.java:745)
@jsjtzyy jsjtzyy changed the title Intermittent HelixBootstrapUpgradeTool failure Intermittent HelixBootstrapUpgradeToolTest failure Apr 2, 2019
@jsjtzyy jsjtzyy closed this as completed Apr 17, 2019
@jsjtzyy
Copy link
Contributor Author

jsjtzyy commented Apr 29, 2019

Fixed in #1162

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pnarayanan pnarayanan

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pnarayanan @jsjtzyy