Support for suspend semantics

[tenzen-y]

What this PR does / why we need it:
I implemented the suspend semantics to Jobs:

• PyTorchJob
• TFJob
• MXJob
• PaddleJob
• MPIJob
• XBGoostJob
• Reconciler.v1

Which issue(s) this PR fixes (optional, in Fixes #<issue number>, #<issue number>, ... format, will close the issue(s) when PR gets merged):
Fixes #1519

Checklist:

• Docs included if any changes are user facing

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: tenzen-y
Once this PR has been reviewed and has the lgtm label, please assign terrytangyuan for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coveralls]

Pull Request Test Coverage Report for Build 5500278893

• 49 of 136 (36.03%) changed or added relevant lines in 15 files are covered.
• 4 unchanged lines in 1 file lost coverage.
• Overall coverage increased (+0.5%) to 33.502%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/controller.v1/common/pod.go	0	1	0.0%
pkg/controller.v1/mpi/mpijob_controller.go	4	5	80.0%
pkg/controller.v1/pytorch/hpa.go	13	15	86.67%
pkg/controller.v1/pytorch/pytorchjob_controller.go	6	9	66.67%
pkg/controller.v1/xgboost/xgboostjob_controller.go	0	4	0.0%
pkg/apis/kubeflow.org/v1/zz_generated.deepcopy.go	0	5	0.0%
pkg/controller.v1/mxnet/mxjob_controller.go	0	5	0.0%
pkg/controller.v1/paddlepaddle/paddlepaddle_controller.go	2	7	28.57%
pkg/reconciler.v1/common/job.go	0	6	0.0%
pkg/apis/kubeflow.org/v1/openapi_generated.go	0	7	0.0%

Files with Coverage Reduction	New Missed Lines	%
pkg/controller.v1/mpi/mpijob_controller.go	4	79.37%

Totals
Change from base Build 5468257726:	0.5%
Covered Lines:	3241
Relevant Lines:	9674

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 5501322598

• 54 of 138 (39.13%) changed or added relevant lines in 15 files are covered.
• 7 unchanged lines in 1 file lost coverage.
• Overall coverage increased (+3.9%) to 36.944%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/controller.v1/common/pod.go	0	1	0.0%
pkg/controller.v1/mpi/mpijob_controller.go	4	5	80.0%
pkg/controller.v1/pytorch/hpa.go	13	15	86.67%
pkg/controller.v1/mxnet/mxjob_controller.go	3	6	50.0%
pkg/controller.v1/pytorch/pytorchjob_controller.go	6	9	66.67%
pkg/controller.v1/paddlepaddle/paddlepaddle_controller.go	4	8	50.0%
pkg/controller.v1/xgboost/xgboostjob_controller.go	0	4	0.0%
pkg/apis/kubeflow.org/v1/zz_generated.deepcopy.go	0	5	0.0%
pkg/reconciler.v1/common/job.go	0	6	0.0%
pkg/apis/kubeflow.org/v1/openapi_generated.go	0	7	0.0%

Files with Coverage Reduction	New Missed Lines	%
pkg/controller.v1/mpi/mpijob_controller.go	7	79.28%

Totals
Change from base Build 5468257726:	3.9%
Covered Lines:	3574
Relevant Lines:	9674

---

💛 - Coveralls

[tenzen-y]

I will create separate PR for each framework since this PR is too big.

[tenzen-y]

/close

[google-oss-prow]

@tenzen-y: Closed this PR.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.