[k8s] Logging SSH Setup on Kubernetes Provision #3266
Conversation
* fix permissions * fix permissions
* Add serve for gemma and fix mixtral dependency * Add hf token * fix model len * Add comment * Serve your private gemma * fix serve yaml * readme * Remove chat completion due to the wrong template * add readme * Update llm/gemma/README.md Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * address comments * Update README.md Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update llm/gemma/README.md Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update llm/gemma/README.md Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Update llm/gemma/README.md Co-authored-by: Zongheng Yang <zongheng.y@gmail.com> * Change to it * Add chat API * use HF_TOKEN env * typo --------- Co-authored-by: Zongheng Yang <zongheng.y@gmail.com>
* when removing cudo credential, sky check fails * remove tips * minor hint fix * fix cluster version for k8s * fix typo
…org#3218) * Add retry for docker pull due to daemon not ready * longer wait time * longer wait time * retry earlier * add retry for retries as well * longer wait time * change wait time * format * Add comment * Fix * Fix indent for azure docker config * Fix docker login config * Fix comments * More robust docker login config * Add retry for docker check * minor fix * Add additional test for stop and start with docker * Fix cancelled
…o hriday/unsloth # Conflicts: # README.md # docs/source/index.rst
sky/provision/kubernetes/instance.py
Outdated
| _preload_content=False, | ||
| _request_timeout=kubernetes.API_TIMEOUT) | ||
| while cmd_output.is_open(): | ||
| cmd_output.update(timeout=1) |
There was a problem hiding this comment.
What's the tradeoff of setting a lower/higher timeout here?
There was a problem hiding this comment.
_request_timeout=kubernetes.API_TIMEOUT is a standard parameter we have defined (https://github.com/shethhriday29/skypilot/blob/a0509c927b4f15c8fb129286b6626beed3fc4e8d/sky/adaptors/kubernetes.py#L22-L23), so just sticking to the default here unless you want otherwise!
There was a problem hiding this comment.
Ah sorry, I mean the timeout in cmd_output.update(timeout=1). What happens if we set it lower/higher?
sky/provision/kubernetes/instance.py
Outdated
| _request_timeout=kubernetes.API_TIMEOUT) | ||
| while cmd_output.is_open(): | ||
| cmd_output.update(timeout=1) | ||
| if cmd_output.peek_stderr(): |
There was a problem hiding this comment.
Can we also print out stdout of the commands run?
There was a problem hiding this comment.
I'm assuming you mean into the provision.log file but also happy to directly print()!
There was a problem hiding this comment.
Thanks @shethhriday29! This makes debugging much easier. Added a few small changes, should be good to go now.
Tested (run the relevant ones):
bash format.shpytest tests/test_smoke.pypytest tests/test_smoke.py::test_fill_in_the_namebash tests/backward_comaptibility_tests.sh