[YUNIKORN-2131] Add ready-to-use example for peemption

[chenyulin0719]

What is this PR for?

Add an immediately usable example to reduce the learning curve for users.

The example shows how to config priority queue and demonstrates below behaviors of preemption:

1. A task cannot preempt tasks outside its preemption fence.
2. Preemption is aiming to ensure that the queue's resource usage reaches at least the guaranteed amount of resources.
3. Preemption can never leave a queue lower than its guaranteed capacity.

What type of PR is it?

• - Bug Fix
• - Improvement
• - Feature
• - Documentation
• - Hot Fix
• - Refactoring

Todos

NA

What is the Jira issue?

https://issues.apache.org/jira/browse/YUNIKORN-2131

How should this be tested?

Follow the example steps in yunikorn-k8shim/deployments/examples/preemption/README.md

Screenshots (if appropriate)

NA

Questions:

NA

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (d2b8ebd) 71.89% compared to head (37a6886) 71.89%.
Report is 2 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #723   +/-   ##
=======================================
  Coverage   71.89%   71.89%           
=======================================
  Files          49       49           
  Lines        7956     7956           
=======================================
  Hits         5720     5720           
  Misses       2039     2039           
  Partials      197      197           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[FrankYang0529]

LGTM. It is a good example to show how to perform preemption. Thanks for the contribution.

[chia7712]

@chenyulin0719 this is a great patch. one small question left.

[chia7712]

Is this max required for the demo? If not, it seems we can remove this queue sandbox

[FrankYang0529]

Yes, it is. Without max resources setting, the available resource depends on users' nodes and it will be hard to determine pod resource requirement.

[chenyulin0719]

Thanks @chia7712 and @FrankYang0529 's review.
Yes, the max resource is required, otherwise the preemption won't be triggered if the queues have enough resources.

[chia7712]

Got it and thanks for sharing. I will test it manually later.

[chia7712]

LGTM