Skip to content

Commit

Permalink
feat: add environment variable for controlling retry iterations
Browse files Browse the repository at this point in the history
  • Loading branch information
TianyiQ committed Dec 29, 2024
1 parent 4d909d9 commit ac4ed32
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion src/abstractions/backends.py
Original file line number Diff line number Diff line change
Expand Up @@ -584,7 +584,8 @@ def sglang_process_batch(
assert len(output) == len(sample_dicts)

count = 0
for _ in range(20):
max_iter = int(os.environ.get("SG_ITER", 20))
for _ in range(max_iter):
bad_indices = [
k
for k in range(len(output))
Expand Down

0 comments on commit ac4ed32

Please sign in to comment.