Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LLM: define common/system-specific event structure #834

Open
lmolkova opened this issue Mar 21, 2024 · 1 comment · May be fixed by #980
Open

LLM: define common/system-specific event structure #834

lmolkova opened this issue Mar 21, 2024 · 1 comment · May be fixed by #980
Assignees
Labels
area:gen-ai enhancement New feature or request

Comments

@lmolkova
Copy link
Contributor

lmolkova commented Mar 21, 2024

Area(s)

llm

Is your change request related to a problem? Please describe.

See #825 (comment)

#825 is going to introduce LLM events and (currently) requires to put prompts and completions as json string in OpenAI format. Some LLM vendors might not use json and instrumentation would be forced to use specific structure.

We should decide if:

  • we define a common format and all LLM vendor instrumentations are forced to use it.
  • we want one event with OpenAI JSON messages format (or another common format) and other vendors are forced to use it.
  • vendor-specific events. E.g. gen_ai.{vendor}.prompt and then vendors could document their own structure or document that they emit OpenAI-formatted events
  • we make format dynamic with additional attributes representing mime type.

This is closely coupled with #829

Describe the solution you'd like

TBD

@lmolkova lmolkova added enhancement New feature or request triage:needs-triage labels Mar 21, 2024
@lmolkova lmolkova changed the title LLM: define common/system-specific structure LLM: define common/system-specific event structure Mar 21, 2024
@lmolkova
Copy link
Contributor Author

Based on the discussion in this thread:

  • we should consider recording individual choices as events rather than completions
  • finish_reason should become a string rather than array and appear on events - this way it matches openai response structure
  • we can still add more attributes or event payload fields to describe the structure of choice

@lmolkova lmolkova self-assigned this Apr 17, 2024
@lmolkova lmolkova linked a pull request Apr 29, 2024 that will close this issue
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:gen-ai enhancement New feature or request
Projects
Development

Successfully merging a pull request may close this issue.

3 participants