Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(machinery): allow base_url of OpenAI to be customized #11467

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat(machinery): allow base_url of OpenAI to be customized #11467

wants to merge 2 commits into from

Conversation

max-wittig
Copy link

@max-wittig max-wittig commented Apr 23, 2024 •
edited

Allows usage of local LLMs or other providers

image

Proposed changes

Adds a new field when setting up the OpenAI integration to allow the user to add a custom base_url

Checklist

  • Lint and unit tests pass locally with my changes.
  • I have added tests that prove my fix is effective or that my feature works.
  • I have added documentation to describe my feature.
  • I have squashed my commits into logic units.
  • I have described the changes in the commit messages.

Other information

@nijel
Copy link
Member

nijel commented Apr 23, 2024

Can you please add documentation which other providers or local LLM work with the OpenAI API and how to configure these?

nijel
nijel reviewed Apr 23, 2024
View reviewed changes
weblate/machinery/openai.py Show resolved Hide resolved
weblate/machinery/forms.py Outdated Show resolved Hide resolved
nijel
nijel reviewed Apr 23, 2024
View reviewed changes
weblate/machinery/forms.py Outdated
widget=forms.Textarea,
help_text=gettext_lazy("Base URL of the OpenAI API"),
required=False,
default=None,
Copy link
Member

@nijel nijel Apr 23, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TypeError: Field.init() got an unexpected keyword argument 'default'

See https://github.com/WeblateOrg/weblate/actions/runs/8800026958/job/24150336022?pr=11467

Seriously, has this been tested?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nijel No, I just pushed it to see the pipeline! Not ready for review! I should have marked it as a draft!

Thanks for the fast review though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend testing locally first against real custom API endpoint. The model selection will need to be changed for custom LLMs as well.

@max-wittig max-wittig marked this pull request as draft April 23, 2024 12:52
@max-wittig max-wittig mentioned this pull request May 1, 2024
Closed
@nijel nijel linked an issue May 23, 2024 that may be closed by this pull request
machinery: support OpenAI API compatible LLMs #11700
Open
@nijel nijel mentioned this pull request May 23, 2024
Open
@max-wittig
feat(machinery): allow base_url of OpenAI to be customized
17b33cf 
Allows usage of local LLMs or other providers
@max-wittig max-wittig force-pushed the feat-allow-customization-of-base-url branch from fd86726 to 17b33cf Compare May 23, 2024 13:21
@max-wittig
Copy link
Author

@nijel I tested it now! In general it would work, but mistral-7b-instruct is not smart enough to deal with your prompt. Other people might have better models hosted though.

@max-wittig max-wittig marked this pull request as ready for review May 23, 2024 13:46
@max-wittig
Copy link
Author

E.g. I tried also using an API client. Same result:

image

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nijel nijel nijel left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

machinery: support OpenAI API compatible LLMs
2 participants
@max-wittig @nijel