add openai spec v0

[aniketmaurya]

This PR allows to expose models through the OpenAI API spec

Here's an example server.py

import litserve as ls


class SimpleLitAPI(ls.LitAPI):
    def setup(self, device):
        self.model = None  # some model

    def decode_request(self, request):
        return request

    def predict(self, x):
        return "foobar"

    def encode_response(self, output):
        return {"text": output}


if __name__ == "__main__":
    api = SimpleLitAPI()
    server = ls.LitServer(api, accelerator="auto", spec=ls.specs.openai.OpenAISpec())
    server.run(port=8000)

and the corresponding client.py:

import requests

response = requests.post("http://127.0.0.1:8000/v1/chat/completions", json={
    "model": "No models available",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  })


print(f"Status: {response.status_code}\nResponse:\n {response.text}")

[lantiga]

Thanks for assembling @aniketmaurya I'm taking this for a spin

[lantiga]

I'm making progress here, slowly but surely :-) I'll be working more on this tomorrow and pass it on, there's still a few kinks I need to fully flash out (feel free to explore of course @aniketmaurya)

[lantiga]

@aniketmaurya if you can look at the test_readme.py failures that would be great

[aniketmaurya]

sure @lantiga, taking a look.

[codecov]

Codecov Report

Attention: Patch coverage is 84.25926% with 51 lines in your changes are missing coverage. Please review.

Project coverage is 81%. Comparing base (bfa9219) to head (936b256).
Report is 1 commits behind head on main.

Additional details and impacted files

@@         Coverage Diff         @@
##           main   #76    +/-   ##
===================================
- Coverage    81%   81%    -0%     
===================================
  Files         8    13     +5     
  Lines       537   750   +213     
===================================
+ Hits        434   606   +172     
- Misses      103   144    +41     

[lantiga]

This now works

from transformers import pipeline
import litserve as ls


class HuggingFaceLitAPI(ls.LitAPI):
    def setup(self, device):
        self.generator = pipeline('text-generation', model='gpt2', device=device)

    def predict(self, prompt):
        out = self.generator(prompt)
        return out[0]["generated_text"]


if __name__ == '__main__':
    api = HuggingFaceLitAPI()
    server = ls.LitServer(api, accelerator='auto', spec=ls.specs.openai.OpenAISpec())
    server.run(port=8000)

import requests

response = requests.post("http://127.0.0.1:8000/v1/chat/completions", json={
    "model": "No models available",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  })

print(f"Status: {response.status_code}\nResponse:\n {response.text}")