[OpenAI] Instructor for Batch Requests

[phatlast96]

Is your feature request related to a problem? Please describe.
Since OpenAI has a bulk feature now and I've been using Instructor's JSON formatting feature a lot, I'd love to combine those 2 since I've been hitting my rate rate limit on the number of requests.

Describe the solution you'd like
I would love use Instructor for JSON formatting of bulk requests. This would be beneficial since it's a lot of cost savings.

Describe alternatives you've considered
Continue hitting my rate limit using Instructor.

Additional context
Add any other context or screenshots about the feature request here.
https://community.openai.com/t/batchapi-is-now-available/718416

[CarloNicolini]

This feature should make instructor even more powerful. Having access to OpenAI batch requests allows a 50% discount. The API is something like this

from openai import OpenAI
client = OpenAI()

client.batches.create(
  input_file_id="file-abc123",
  endpoint="/v1/chat/completions",
  completion_window="24h"
)

Clearly there is some difference to be supported, but we'd all love the batch feature to be implemented by instructor ♥️

[ivanleomk]

@CarloNicolini and @phatlast96 what do you think of an api like this?

from instructor.batch import BatchJob
from pydantic import BaseModel
import enum

emails = [
    "Hello there I'm a Nigerian prince and I want to give you money",
    "Meeting with Thomas has been set at Friday next week",
    "Here are some weekly product updates from our marketing team",
]

messages = [
    [
        {
            "role": "system",
            "content": f"Classify the following email {email}",
        }
    ]
    for email in emails
]


class Labels(str, enum.Enum):
    """Enumeration for single-label text classification."""

    SPAM = "spam"
    NOT_SPAM = "not_spam"


class SinglePrediction(BaseModel):
    """
    Class for a single class label prediction.
    """

    class_label: Labels


with open("./test.jsonl", "w") as f:
    for line in BatchJob.create_from_messages(
        messages_batch=messages,
        model="gpt-3.5-turbo",
        response_model=SinglePrediction,
    ):
        f.write(line + "\n")

This should be flexible enough for you to use in a variety of different situations. Also implemented a cli tool that allows you to then create these batch jobs with a syntax like

instructor batch create-from-file --file-path ./test.jsonl
                                        OpenAI Batch Jobs
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┓
┃ Batch ID                       ┃ Created At          ┃ Status    ┃ Failed ┃ Completed ┃ Total ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━┩
│ batch_HcjI2wG46Y1LYGw3ySfyT0eQ │ 2024-06-12 15:45:37 │ completed │ 0      │ 3         │ 3     │
│ batch_YiRKLAmKBhwxM8g8C3yLBoVD │ 2024-06-12 15:09:44 │ completed │ 0      │ 3         │ 3     │
│ batch_hS0XGlXzTVS7SxndUPUo2fdj │ 2024-06-12 15:05:59 │ completed │ 0      │ 3         │ 3     │
│ batch_6s4FmcaV7woamWBprg8WhCPm │ 2024-06-12 14:26:34 │ completed │ 0      │ 3         │ 3     │
└────────────────────────────────┴─────────────────────┴───────────┴────────┴───────────┴───────┘