`getModelJSON` fails with a bundler (see reproduction)

[maudnals]

Describe the bug
Hi there, I'm trying to use remote models in a basic web app that relies on a bundler (parcel).getModelJSON doesn't seem to load models correctly. I put together a minimal reproduction, see below.
(Note: no issue observed when not using a bundler, and e.g. using CDN transformers instead)
This may be a duplicate of, or related to, #364.
Thanks!

How to reproduce
Bug repro repo: https://github.com/maudnals/bug-repro-transformersjs/tree/main
Steps to reproduce:

• Clone this repo and navigate to its root
• Run yarn install
• Run yarn parcel index.html
• Observe the code in script.js. We're simply calling await pipeline('sentiment-analysis')
• Open http://localhost:1234 in your browser, or whichever URL parcel gives you
• Open DevTools
• Observe the error logged in the DevTools console upon model loading:

Uncaught (in promise) SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON
    at JSON.parse (<anonymous>)
    at getModelJSON (hub.js:551:17)
    at async Promise.all (:61216/index 0)
    at async loadTokenizer (tokenizers.js:52:16)
    at async AutoTokenizer.from_pretrained (tokenizers.js:3826:48)
    at async Promise.all (:61216/index 0)
    at async loadItems (pipelines.js:2193:5)
    at async pipeline (pipelines.js:2139:19)
    at async app.js:9:14

Expected behavior
The model should load properly.

Logs/screenshots

Uncaught (in promise) SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON
    at JSON.parse (<anonymous>)
    at getModelJSON (hub.js:551:17)
    at async Promise.all (:61216/index 0)
    at async loadTokenizer (tokenizers.js:52:16)
    at async AutoTokenizer.from_pretrained (tokenizers.js:3826:48)
    at async Promise.all (:61216/index 0)
    at async loadItems (pipelines.js:2193:5)
    at async pipeline (pipelines.js:2139:19)
    at async app.js:9:14

Environment

• Transformers.js version: "@xenova/transformers": "^2.6.2"
• Browser (if applicable): Chrome 118
• Operating system (if applicable): MacOS

Addtional context
When adding a breakpoint in getModelJSON, the model path seem fine (openai/...).
I wonder if the issue lays in path rewriting somewhere alongside the bundling steps?

[xenova]

Hi there 👋 Thanks for making the repro 👍

I believe the problem is the following:

1. By default, transformers.js (like the python library) checks your local server for the model files.
2. If the model files are found locally, we use those.
3. If not, the model files are loaded from the HF hub.
4. The way we detect whether the files are found locally or not, is by looking at the status code being 404. If this is not the case, the file is assumed to be found locally, and the JSON parsing occurs (in this case, for the config.json, tokenizer.json, or tokenizer_config.json file)
5. I haven't tested with the bundler you seem to be using ("Parcel" I think it's called?), but I think that it's returning some HTML page for the 404 page (hence the error being unable to parse JSON due to unexpected character).

Could you try the following:

1. Start the server and visit http://localhost:1234/models/Xenova/distilbert-base-uncased-finetuned-sst-2-english. This should fail and result in a 404 page. Could you open dev tools to see what the error code or status of the page is? It should be 404 too.
2. Update script.js to include the following:

   import { pipeline, env } from '@xenova/transformers';
   env.allowLocalModels = false;

This will bypass the local file check and download from the HF hub instead.

[pappitti]

Hi there, facing this exact issue at the moment. I thought about disallowing local models and I am glad to see it suggested here but it does not work for me : I keep getting the error.

[kungfooman]

I digged a bit into the generated Parcel code and the source map support is kind of broken (making it hard to inspect values and setting breakpoints), but from a bit of testing I figured that this works at least:

import { pipeline, env } from '@xenova/transformers';
env.allowLocalModels = false;
env.useBrowserCache = false;

Entire test script:

script.js

import { pipeline, env } from '@xenova/transformers';
env.allowLocalModels = false;
env.useBrowserCache = false;
console.log('✅ Pipeline function is correctly loaded: \n', pipeline);
console.log('❌ But attempting to create a pipeline fails: ');
// Allocate a pipeline for sentiment-analysis
(async function () {
  const pipe = await pipeline('sentiment-analysis');
  console.log("pipe", pipe);
})();

[pappitti]

Thanks, I got to the same conclusion : it only works if you force the download form the remote URL. I could not get to the bottom of it.

I was hopeful that another solution existed because, for the project I am working on, this work-around means that users would download the model over and again

[cipherphage]

I digged a bit into the generated Parcel code and the source map support is kind of broken (making it hard to inspect values and setting breakpoints), but from a bit of testing I figured that this works at least:

import { pipeline, env } from '@xenova/transformers';
env.allowLocalModels = false;
env.useBrowserCache = false;

Entire test script:

script.js

import { pipeline, env } from '@xenova/transformers';
env.allowLocalModels = false;
env.useBrowserCache = false;
console.log('✅ Pipeline function is correctly loaded: \n', pipeline);
console.log('❌ But attempting to create a pipeline fails: ');
// Allocate a pipeline for sentiment-analysis
(async function () {
  const pipe = await pipeline('sentiment-analysis');
  console.log("pipe", pipe);
})();

This really helped me! I know it's only tangentially related to the original post, but I ran into the same JSON parsing error when trying to build the client-side example from Hugging Face's Next.JS tutorial (https://huggingface.co/docs/transformers.js/tutorials/next#client-side-inference) and adding the line env.useBrowserCache = false; to my worker is what fixed it for me. After that, I had no issues using the library.

[pappitti]

I don’t think it is only tangentially related, I think it is the same bug everywhere : before downloading the tokenizer from HF, transformers.js checks if it exists locally or in browser cache by following a specific path for each option. If the
path does not exist, a 404 error is thrown. I think that the option (local or cache) is supposed to be ignored in case of 404 error but it isn’t. At one point the problematic path is just replaced by the root of your app (typical fall back) and this is why an HTML page is parsed instead of a json.
What I need to figure out now is when (at which point) the root is swapped in

[cipherphage]

@pappitti I see what you mean. Here's more context and details I found for my particular case:

Attempting to run in development mode the client-side example provided in Hugging Face's Nextjs tutorial:

• getModelJSON parsing error is thrown on initial load of the web worker.
• This is specifically on Brave Version 1.61.101 Chromium: 120.0.6099.71 (Official Build) (x86_64).
• This does NOT happen on Chrome Version 120.0.6099.62 (Official Build) (x86_64); it runs without errors there (my apologies for not trying Chrome previously).
• Here is the full screenshot of the error, including the warnings that lead up to it:

[wesbos]

Also hitting this, but it seems V3 will fix it.

Work around for now:

env.allowLocalModels = false;
env.useBrowserCache = false;

seems to be fixing it

[azer]

Also hit this issue and spent some time debugging. Several things contributing to this issue:

• Making request to local url instead of remoteURL
• Caching bad response with local key
• In the next requests, looking up cache by both local and remote urls
• Not invalidating the cache after receiving bad response

[m4a1carbin4]

I Also hitting this, and In my debugging

I think cache.match(name) Seems to get data from an abnormal url.

this is what cache.match(name) return data and I don't know exactly, but it's confirmed that url is " "

In addition, I Think there is a problem that is likely to be a dev environmental problem in next.js with high confirmation, the build result works normally in the space of huggingface.

[maudnals]

Quick follow-up for the record, as it seems others have been encountering the issue:

• Can confirm that env.allowLocalModels = false; with env.useBrowserCache = false; fixes it for me, at least with vite. It does slow down development since the model has to be downloaded from scratch at every load.
• Great to hear this may be fixed in V3!