Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: vllm v0.6.2/v0.6.3 is easy to generate random output if there are many symbols(not words) in prompt #9409

Closed
1 task done
tonyaw opened this issue Oct 16, 2024 · 2 comments
Labels
bug Something isn't working stale

Comments

@tonyaw
Copy link

tonyaw commented Oct 16, 2024

Your current environment

The output of `python collect_env.py`
Your output of `python collect_env.py` here

Model Input Dumps

No response

🐛 Describe the bug

I have an observation:
If there are many symbols(non-words) in prompt, like following cases, v0.6.x is much easier to generate random output:

  • markdown table symbo "|"
  • yaml in json value
  • json in json

One output is as following:

I have been changed to be | Mandatory. |  |  |  |  |  |  |  |  |  |  |  | Mandatory (SIP...-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. |"

If I switch back to 0.5.4, this issue will be gone.
Could you please help to guide me the possible reason and how to avoid it?
Thanks in advance!

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
@tonyaw tonyaw added the bug Something isn't working label Oct 16, 2024
Copy link

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

@github-actions github-actions bot added the stale label Jan 15, 2025
Copy link

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Feb 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working stale
Projects
None yet
Development

No branches or pull requests

1 participant