Singularity should be able to handle registration, re-registration, and leadership changes without aborting

[wsorenson]

No description provided.

[wsorenson]

Some complications:

Losing leadership:

• Mesos sends an error when framework fails over - in the past, there were some cases where this was the only way we found out about a framework failing over (so we needed to stop leadership)
• Deletion of leader latch doesn't re-create the leader latch

Uncaught exceptions:

• Probably could handle, but I would feel more comfortable here if we we had more control over the SchedulerDriver, the issue is right now some of our aborting is a hedge against mesos java bugs.

[tpetr]

@wsorenson thoughts on tackling this? or are we happy with our abort strategy?

[stevenschlansker]

End-user opinion: the aborts are worrying but once you learn to stop fretting they really do not present any sort of urgent problem to us.

[ssalinas]

fixed in #2032