Adding operation name to LLM spans #995
Conversation
Signed-off-by: sudivate <sudivate@microsoft.com>
Signed-off-by: sudivate <sudivate@microsoft.com>
… into sudivate/consistent-span-names Signed-off-by: sudivate <sudivate@microsoft.com>
Signed-off-by: sudivate <sudivate@microsoft.com>
model/registry/gen-ai.yaml
Outdated
| stability: experimental | ||
| type: string | ||
| brief: The name of the LLM operation request being made. | ||
| examples: ['chat.completions', 'embeddings', 'speech.generations', 'audio.transcriptions', 'audio.translations', 'image.generations'] |
There was a problem hiding this comment.
| examples: ['chat.completions', 'embeddings', 'speech.generations', 'audio.transcriptions', 'audio.translations', 'image.generations'] | |
| examples: ['chat', 'embeddings', 'speech.generations', 'audio.transcriptions', 'audio.translations', 'image.generations'] |
There was a problem hiding this comment.
I guess if we convert it into a enum, it would be both - chat and completions (legacy)?
There was a problem hiding this comment.
True, ideally it should be completions and chat.completions
There was a problem hiding this comment.
I find chat.completions to be highly confusing with completions. I'd stick with chat and completions
lmolkova
changed the title
model/registry/gen-ai.yaml
Outdated
| stability: experimental | ||
| type: string | ||
| brief: The name of the LLM operation request being made. | ||
| examples: ['chat.completions', 'embeddings', 'speech.generations', 'audio.transcriptions', 'audio.translations', 'image.generations'] |
There was a problem hiding this comment.
I guess if we convert it into a enum, it would be both - chat and completions (legacy)?
|
|
||
| | Value | Description | Stability | | ||
| | ---------------------- | -------------------- | ---------------------------------------------------------------- | | ||
| | `completions` | Completions |  | |
There was a problem hiding this comment.
Howa about adding tokenize as well?
tokenize(prompt, return_tokens=False)[source]
The text tokenize operation allows you to check the conversion of provided input to tokens for a given model. It splits text into words or sub-words, which then are converted to ids through a look-up table (vocabulary). Tokenization allows the model to have a reasonable vocabulary size.
|
We can potentially continue the discussion here after #955 is merged |
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
Fixes #
Changes
Adding operation name to LLM spans
Note: if the PR is touching an area that is not listed in the existing areas, or the area does not have sufficient domain experts coverage, the PR might be tagged as experts needed and move slowly until experts are identified.
Merge requirement checklist
[chore]