Add support for guided decoding (fixes #288)

[br3no]

This pull request extends the api_server.py with optional guided decoding with regex and JSON schema resolving issue #288.

The API is extended with the optional parameters regex and schema. The functionality is opt-in and needs to be activated by the CLI parameter --guided-decoding-engine that defaults to None and accepts the value outlines.

The new module guided_decoding.py implements the integration of the outlines logits processors.

This pull request also changes the API of the vLLM LogitsProcessors to add a sequence id for each sequence, to support stateful logits processors.

Starting the api_server with the extra CLI arguments --guided-decoding-engine outlines allows one to issue these kinds of requests:

response = requests.post("http://hal9000:1984/generate", json={
    "prompt" : "The best language for type-safe systems programming is ",
    "regex" : "(Python|Java|C|C\+\+|C#|JavaScript|PHP|Swift|Go|Ruby|TypeScript|Kotlin|Rust)",
    "max_tokens" : 10,
    "n" : 2
})
response.json()

Producing

 {'text': ['The best language for type-safe systems programming is C++',
  'The best language for type-safe systems programming is Go']}

response = requests.post("http://hal9000:1984/generate", json={
    "prompt" : """Return a json object for the following schema: {"type": "object", "properties": {"name": {"type": "string", "maxLength" : 20}, "age": {"type": "integer"}}}""",
    "schema" : {"type": "object", "properties": {"name": {"type": "string", "maxLength" : 20}, "age": {"type": "integer"}}}
})
response.json()

Producing

{'text': ['Return a json object for the following schema: {"type": "object", "properties": {"name": {"type": "string", "maxLength" : 20}, "age": {"type": "integer"}}}: {"name": "John", "age": 30}']}

The changes here are partially inspired by the integration of vLLM in outlines (https://github.com/outlines-dev/outlines/blob/main/outlines/serve/serve.py).

[esmeetu]

Hi, @br3no
What's the difference between https://github.com/noamgat/lm-format-enforcer and outlines? IMK, They could be done json and regex structure output.

[simon-mo]

I think outlines has lower runtime overhead as compared to lmformatenforcer.

[br3no]

@esmeetu I actually have a branch with support for lm-format-enforcer. I chose not to add it to this PR because I couldn't yet find a way to reach reasonable speed. You can follow the discussion here: noamgat/lm-format-enforcer#65 (comment)

[br3no]

@simon-mo let me know if you feel this goes in the right direction and if there is anything I can do to help in the review process.

[simon-mo]

Hi @br3no, thank you sooo much for this PR. However, I'm bummed to say that we are not going to add more functionality to the simple api server, rather focusing the complex features in open ai compatible server. I think we will end up merging #2819 (which is based off your commit and you are one of the co-author!). Any review on that PR is appreciated.

[br3no]

Hi @simon-mo, no worries. I'll chime in on issue #2819.