Allow embedding model fields, fix coupled model fields, add custom OpenAI provider #1264

srdas · 2025-02-27T07:17:11Z

Description

Fixes Add support for embedding models served through an OpenAI API #1240.
Fixes Bug: Model field inputs are coupled in Settings UI #1261
Related to PR Simplifying the OpenAI provider to use multiple model providers #1248 (closed to open this extended one)

The OpenAI model interface has been widely adopted by many model providers (DeepSeek, vLLM, etc.) and this PR enables accessing these models using the OpenAI provider. Current OpenAI models are also accessible via the same interface.

This PR also updates related documentation on the use of these models that work via the OpenAI provider.

These updates work for selecting chat and embeddings models. Chat models are tested to work with models from OpenAI, DeepSeek, and models hosted by vLLM. Embedding models are tested for OpenAI models. DeepSeek does not have an API for embedding models, and OpenRouter also does not support as yet any embedding models.

Also, this PR corrects the coupled fields problem in the AI Settings page.

Finally, added the embedding fields to the config_scheme.json and made related changes to config_manager.py and test_ config_manager.py

Each of these changes is now described below in some more detail.

Demo of new features

See the new usage of models and the required settings shown below, note the new "OpenAI::general interface":

For any OpenAI model:

For DeepSeek models:

For models deployed with vLLM:

Embedding Models

First, tested to make sure that the OpenAI models are working as intended with no changes to the code:

Second, modified check that the interface takes any OpenAI embedding model as an input and test that it works with OpenAI models as before:

Fixed coupled model field inputs

We can see that the fields are not coupled any more as shown below:

Added `embeddings_fields` to `config_schema.json`

Updated config_manager.py to handle the new fields.
Also updated analogous code for continuation models.
Updated test_config_manager.py for the additional embedding field in config.

for more information, see https://pre-commit.ci

dlqqq

@srdas Thank you for working on this! This is definitely one of the most challenging tasks that you've taken on thus far. Left some feedback for you below.

I think it would be best to hold off on merging this PR until after the v2.30.0 release scheduled for tomorrow. The ConfigManager areas of the code are fragile, and making lots of changes there introduces risk to users. I recommend that we ship this later to give us more time to thoroughly test these changes and mitigate that risk.

packages/jupyter-ai/jupyter_ai/config_manager.py

packages/jupyter-ai/src/components/settings/use-server-info.ts

packages/jupyter-ai/src/components/chat-settings.tsx

for more information, see https://pre-commit.ci

eugenecherepanov · 2025-03-17T13:31:59Z

do you have any plan for merge this feature?

srdas · 2025-03-17T14:59:56Z

do you have any plan for merge this feature?

@eugenecherepanov I have a couple of things still not working that need to be cleared before this can be fully tested and reviewed, hoping to get it done very soon.

dlqqq

@srdas The code now looks much cleaner, thank you for contributing this! Verified the changes locally.

Note that there is a bug where if you have unsaved chat model fields, the fields get reset if you change either the embedding or language model. This is an existing bug however, so this can be addressed separately in the future.

I left 2 additional comments below for you to address, but I will approve this now since those comments are minor.

dlqqq · 2025-03-19T21:29:51Z

packages/jupyter-ai-magics/jupyter_ai_magics/partner_providers/ollama.py

+        "*",
+        # "nomic-embed-text",
+        # "mxbai-embed-large",
+        # "all-minilm",
+        # "snowflake-arctic-embed",


I'm fine with requiring Ollama embedding model users to set the model IDs manually, since we are doing that for the Ollama chat model anyways.

However, can you remove the comments and add a help attribute to show a help message?

dlqqq · 2025-03-19T22:14:30Z

packages/jupyter-ai/src/components/chat-settings.tsx

+                // .filter(em => em !== '*') // TODO: support registry providers
+                // .filter(em => emp.embedding_models.includes(em))


Can you also remove this comment?

srdas · 2025-03-20T17:44:12Z

@meeseeksdev please backport to 2.x

…d model fields, add custom OpenAI provider

…upled model fields, add custom OpenAI provider) (#1282) * Backport PR #1264: Allow embedding model fields, fix coupled model fields, add custom OpenAI provider * Fix CI --------- Co-authored-by: Sanjiv Das <[email protected]>

…ds, fix coupled model fields, add custom OpenAI provider) (jupyterlab#1282) * Backport PR jupyterlab#1264: Allow embedding model fields, fix coupled model fields, add custom OpenAI provider * Fix CI --------- Co-authored-by: Sanjiv Das <[email protected]>

srdas mentioned this pull request Feb 27, 2025

Simplifying the OpenAI provider to use multiple model providers #1248

Closed

srdas added the enhancement New feature or request label Feb 27, 2025

srdas changed the title ~~Create a custom OpenAI provider to use multiple model providers~~ Create a custom OpenAI provider to use multiple models Feb 27, 2025

srdas marked this pull request as ready for review February 27, 2025 19:57

dlqqq linked an issue Feb 27, 2025 that may be closed by this pull request

Add support for embedding models served through an OpenAI API #1240

Closed

This was referenced Mar 1, 2025

Bug: Model field inputs are coupled in Settings UI #1261

Closed

Refactor Chat Handlers to Simplify Initialization #1257

Merged

srdas and others added 10 commits March 4, 2025 22:16

Simplifying the OpenAI provider to use multiple model providers

f02f654

Update openrouter.md

2cfd037

[pre-commit.ci] auto fixes from pre-commit.com hooks

1beac32

for more information, see https://pre-commit.ci

openai general interface added

3999a7e

[pre-commit.ci] auto fixes from pre-commit.com hooks

866adbd

for more information, see https://pre-commit.ci

embedding

291a76d

[pre-commit.ci] auto fixes from pre-commit.com hooks

214a664

for more information, see https://pre-commit.ci

Updated settings to take OpenAI generic embedding models

175cdfb

added openai generic embeddings screenshot

b33f4e2

Fixed Issue 1261

f59026b

srdas force-pushed the openai_generic_2 branch from 1ba53fd to f59026b Compare March 5, 2025 06:20

srdas requested a review from dlqqq March 5, 2025 15:38

srdas and others added 4 commits March 7, 2025 09:33

bump version floor on jupyter server

245bdb1

linter

002f6bb

adding embedding model fields

e66c0c1

[pre-commit.ci] auto fixes from pre-commit.com hooks

f5ca39d

for more information, see https://pre-commit.ci

srdas changed the title ~~Create a custom OpenAI provider to use multiple models~~ Create a custom OpenAI provider to use multiple models, resolve coupled input fields, add Embedding Fields to config Mar 10, 2025

srdas added 6 commits March 10, 2025 08:57

Update test_config_manager

12b0e18

Update pyproject.toml

448bd41

Update pyproject.toml

700e0a6

Update pyproject.toml

a3b7a18

pyproject.toml fixes

434d6fe

pyproject.toml updates

1d77b09

srdas added 3 commits March 11, 2025 14:15

pyproject toml updates

c54a364

Merge branch 'main' into openai_generic_2

81ced73

update snapshot

311e9a8

dlqqq requested changes Mar 12, 2025

View reviewed changes

packages/jupyter-ai/jupyter_ai/config_manager.py Outdated Show resolved Hide resolved

packages/jupyter-ai/src/components/settings/use-server-info.ts Outdated Show resolved Hide resolved

packages/jupyter-ai/src/components/chat-settings.tsx Outdated Show resolved Hide resolved

srdas and others added 5 commits March 14, 2025 01:24

writing config file correctly

439f8ea

[pre-commit.ci] auto fixes from pre-commit.com hooks

04e6477

for more information, see https://pre-commit.ci

tsx lint

b1ba3a3

Update use-server-info.ts

b03a173

Update pyproject.toml

f9103e9

srdas added 3 commits March 17, 2025 10:01

adds embedding_models attribute

f6c7adb

Fixed display of Base url for embeddings and completions

5027726

removed embedding_models

8176b3d

srdas requested review from dlqqq and keerthi-swarna March 17, 2025 23:50

dlqqq approved these changes Mar 19, 2025

View reviewed changes

dlqqq changed the title ~~Create a custom OpenAI provider to use multiple models, resolve coupled input fields, add Embedding Fields to config~~ Add embedding models, fix coupled model inputs, add custom OpenAI provider Mar 19, 2025

dlqqq added the bug Bugs reported by users label Mar 19, 2025

dlqqq changed the title ~~Add embedding models, fix coupled model inputs, add custom OpenAI provider~~ Allow embedding model fields, fix coupled model fields, add custom OpenAI provider Mar 19, 2025

srdas added 3 commits March 19, 2025 17:48

Added help fields

0b78606

Update chat-settings.tsx

3cb32be

minor reversions moved to new issue

406f985

srdas merged commit d4dcef9 into jupyterlab:main Mar 20, 2025
9 checks passed

meeseeksmachine pushed a commit to meeseeksmachine/jupyter-ai that referenced this pull request Mar 20, 2025

Backport PR jupyterlab#1264: Allow embedding model fields, fix couple…

143606c

…d model fields, add custom OpenAI provider

meeseeksmachine mentioned this pull request Mar 20, 2025

Backport PR #1264 on branch 2.x (Allow embedding model fields, fix coupled model fields, add custom OpenAI provider) #1282

Merged

dlqqq mentioned this pull request Mar 25, 2025

Migrate old config schemas, fix v2.31.0 regression #1294

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Allow embedding model fields, fix coupled model fields, add custom OpenAI provider #1264

Allow embedding model fields, fix coupled model fields, add custom OpenAI provider #1264

Uh oh!

srdas commented Feb 27, 2025 •

edited

Loading

Uh oh!

dlqqq left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eugenecherepanov commented Mar 17, 2025

Uh oh!

srdas commented Mar 17, 2025

Uh oh!

dlqqq left a comment

Uh oh!

dlqqq Mar 19, 2025

Uh oh!

dlqqq Mar 19, 2025

Uh oh!

Uh oh!

srdas commented Mar 20, 2025

Uh oh!

Uh oh!

		// .filter(em => em !== '*') // TODO: support registry providers
		// .filter(em => emp.embedding_models.includes(em))

Uh oh!

Allow embedding model fields, fix coupled model fields, add custom OpenAI provider #1264

Allow embedding model fields, fix coupled model fields, add custom OpenAI provider #1264

Uh oh!

Conversation

srdas commented Feb 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Demo of new features

Embedding Models

Fixed coupled model field inputs

Added embeddings_fields to config_schema.json

Uh oh!

dlqqq left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eugenecherepanov commented Mar 17, 2025

Uh oh!

srdas commented Mar 17, 2025

Uh oh!

dlqqq left a comment

Choose a reason for hiding this comment

Uh oh!

dlqqq Mar 19, 2025

Choose a reason for hiding this comment

Uh oh!

dlqqq Mar 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

srdas commented Mar 20, 2025

Uh oh!

Uh oh!

srdas commented Feb 27, 2025 •

edited

Loading

Added `embeddings_fields` to `config_schema.json`