[Bugfix] Guided decoding falls back to outlines when fails to import xgrammar

[terrytangyuan]

This fixes the issue when xgrammar module cannot be imported successfully for some reason, e.g. triton is not available. This fallback allows users to still use guided decoding when xgrammar cannot be used.

  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/client.py", line 606, in _process_request
    params = await \
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/async_llm_engine.py", line 553, in build_guided_decoding_logits_processor_async
    processor = await get_guided_decoding_logits_processor(
  File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/guided_decoding/__init__.py", line 109, in get_guided_decoding_logits_processor
    return get_local_xgrammar_guided_decoding_logits_processor(
  File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/guided_decoding/xgrammar_decoding.py", line 38, in get_local_xgrammar_guided_decoding_logits_processor
    config = GrammarConfig.from_guided_params(guided_params=guided_params,
  File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/guided_decoding/xgrammar_decoding.py", line 174, in from_guided_params
    tokenizer_data = TokenizerDataCache.get_tokenizer_data(tokenizer)
  File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/guided_decoding/xgrammar_decoding.py", line 87, in get_tokenizer_data
    vocab_type = xgr.VocabType.RAW
NameError: name 'xgr' is not defined

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[terrytangyuan]

This is easier to review when hiding the whitespaces: https://github.com/vllm-project/vllm/pull/12976/files?diff=unified&w=1

[russellb]

just some minor style feedback. thanks for the fix!

[russellb]

lgtm, thanks!

[terrytangyuan]

Ci failures seem unrelated: requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/openai-community/gpt2/tree/main?recursive=True&expand=False

@mgoin Would you like to merge this manually?