Contact person: Fix ONNX Init Errors

Phone:Show

Email:Show

Location: Manhattan, United States

Budget: Recommended by industry experts

Time to start: As soon as possible

Project description:
"Project Description

We have a browser app (FastAPI + Uvicorn backend serving static files) that imports an ONNX model (flan-t5-small) into IndexedDB, then attempts to load it using [login to view URL] v2.x with the onnxruntime-web (WASM) backend.

Model import + validation works.

WASM binaries are served correctly (200, correct MIME, proper magic bytes).

IndexedDB VFS works (all model files available).

Environment is cross-origin isolated, SharedArrayBuffer available.

Problem: Pipeline initialization fails with no available backend found across all modes (auto, onnx, wasm, SIMD and non-SIMD).

We need a developer to investigate and fix the backend initialization so that the pipeline loads and inference runs inside the browser.

Requirements / Skills Needed

You must have hands-on experience with the following (don’t apply unless you’ve actually solved similar issues before):

[login to view URL] (v2.x) internals (especially model loading and device/backend selection).

onnxruntime-web (WASM and SIMD variants).

Debugging browser-based ML pipelines (WebAssembly, IndexedDB, SharedArrayBuffer).

Strong knowledge of JavaScript/TypeScript, async loading, bundling/minification issues.

Understanding of browser storage APIs (IndexedDB, CacheStorage) and custom fetch/VFS implementations.

Debugging WASM runtime initialization errors (LinkError, TypeError: reading 'buffer', etc.).

Ability to interpret minified stack traces and trace them back to library internals.

Experience configuring cross-origin isolation (COOP/COEP) and serving WASM files with proper MIME headers.

Comfort working with FastAPI/Uvicorn static serving (not the main focus, but you’ll need to test in this environment).

Deliverables

Identify why ORT backends are not being recognized by [login to view URL] v2.x.

Fix the issue so the ONNX model initializes and runs inference in-browser.

Provide a short technical write-up of what was wrong and what was changed.

Nice-to-Haves

Prior work with Hugging Face [login to view URL] on custom deployments.

Familiarity with WebGPU / WebNN backends (future upgrade path).

Experience with cross-platform debugging (Windows, Linux VPS).

What to Include in Your Proposal

A brief explanation of your relevant experience with [login to view URL] and onnxruntime-web.

Examples of browser-based ML projects you’ve debugged/fixed.

Confirmation you can debug WASM backend initialization errors.

Note: This is not a beginner-friendly task. If you’ve never debugged [login to view URL] + ONNX WASM before, please don’t apply.


Detailed technical explanation:

Technical incident report: browser model fails to initialize (“no available backend found”)
Summary

After successfully importing the ONNX model into IndexedDB, initializing the text generation pipeline in the browser fails with repeated no available backend found errors originating inside transformers.min.js. This occurs across multiple device hints (auto, onnx, wasm) and both SIMD and non-SIMD modes. WASM binaries are reachable (200 + correct MIME), cross-origin isolation is in place (SAB available), and the model files are present and fetchable via the custom IndexedDB VFS. The failure happens at the point where ONNX Runtime (ORT) is created/used by Transformers.js.

Environment

OS / shell: Windows (PowerShell paths), Python 3.11 venv

Server: FastAPI + Uvicorn at http://localhost:9000

COOP/COEP/CORP headers set (cross-origin isolated)

Static assets served from /static/lib

Browser: Chromium-based (DevTools present; SharedArrayBuffer available)

[login to view URL]: Pinned to v2.x (2.14.1 was used; bundle does not include “webgpu” strings)

ONNX Runtime Web (WASM): 1.17.x binaries deployed in /static/lib

[login to view URL] (200, application/wasm)

[login to view URL] (200, application/wasm)

Magic bytes verified: [0,97,115,109]

Model: flan-t5-small ONNX (zip download → unzipped into IndexedDB)

Encoder ONNX: ~141 MB

Decoder ONNX: ~232 MB

[login to view URL], [login to view URL], [login to view URL], [login to view URL], generated [login to view URL]

Storage: IndexedDB store model-store/files with chunking for large files; import validation shows all required files present

Configuration that leads to failure

ORT configured non-threaded on the main thread:

[login to view URL] = "wasm"

[login to view URL] = false

[login to view URL] = 1

[login to view URL] = "http://localhost:9000/static/lib/"

[login to view URL] either cleared or pinned to the two non-threaded binaries ([login to view URL], [login to view URL])

[login to view URL] toggled between true and false during attempts

Transformers flags:

[login to view URL] = true

[login to view URL] = false

VFS fetch shim redirects indexeddb://flan-t5-small/... to IndexedDB blobs

Reproduction steps

Start server: uvicorn main:app --host 0.0.0.0 --port 9000 --reload.

Open http://localhost:9000 in Chromium-based browser.

Click “Download & Import Model”.

Observe logs:

“Unzipping to memory…”

“Writing files to IndexedDB…” (large ONNX files written)

“Imported 5 files (359.2 MB). Validating…”

“Validation result: true (…)”

Pipeline initialization runs automatically and fails.

Observed logs (representative)

Model import / validation

[APP] STATUS: Imported 5 files (359.2 MB). Validating…
[APP] Validation result: true (7) [...]


Pipeline attempts (Transformers v2.x device permutations)

[APP] ORT current config: { proxy: false, simd: true, numThreads: 1, wasmPrefix: 'http://localhost:9000/static/lib/', ... }

[APP] pipeline attempt: auto { dtype: 'fp32' }
[APP] pipeline attempt failed: auto Error: no available backend found. ERR: [wasm] TypeError: Cannot read properties of undefined (reading 'buffer')

[APP] pipeline attempt: onnx { dtype: 'fp32', device: 'onnx' }
[APP] pipeline attempt failed: onnx Error: no available backend found. ERR:

[APP] pipeline attempt: wasm { dtype: 'fp32', device: 'wasm' }
[APP] pipeline attempt failed: wasm Error: no available backend found. ERR:


Fallback loop (SIMD → BASIC)

pipeline-load: 1771 ms
Pipeline load failed in mode=simd: Error: no available backend found. ERR: [...]
[APP] ORT current config: { proxy: false, simd: false, numThreads: 1, ... }
pipeline-load: 1828 ms
Pipeline load failed in mode=basic: Error: no available backend found. ERR: [...]
[APP] Error: No available backend succeeded. Last error: no available backend found. ERR:


Earlier (during other trials)

With threaded worker + newer stacks: LinkError: [login to view URL](): Import #81 "a" "Ca": function import requires a callable

With v2.x SIMD path: TypeError: Cannot read properties of undefined (reading 'buffer')

Additional checks already performed

VFS sanity: [login to view URL]('indexeddb://flan-t5-small/[login to view URL]'|'[login to view URL]'|'[login to view URL]') → 200 OK

WASM availability: HEAD on [login to view URL] and [login to view URL] → 200, application/wasm

Magic bytes: first 4 bytes of both wasm files → 00 61 73 6d

Cross-origin isolation: [login to view URL] === true, SharedArrayBuffer defined

Global ORT: [login to view URL] is false (Transformers bundles ORT internally)

Model presence: IndexedDB shows all expected keys and sizes; validation logs true

Where the failure occurs

From the stack traces, the failure arises inside the minified [login to view URL] during ONNX Runtime initialization / session creation:

[login to view URL]: s(...) → [login to view URL](...) → T(...) → ct.from_pretrained(...) → Ti.from_pretrained(...)
→ R(...) → tryLoadPipelineOnce(...)


The “reading 'buffer'” error happens on the SIMD path in v2.x.

Non-SIMD BASIC path also fails, but the error detail collapses to “no available backend found” without additional message.

The failures occur after successful model import, and without any network 404 or MIME issues for wasm or small model files.

Notable characteristics

The app uses a custom fetch shim to serve indexeddb:// URLs from IndexedDB; this is working for small JSON assets and binary chunks reassembled from parts. Large .onnx blobs are retrieved as Blob instances and handed back via Response objects.

[login to view URL] = false to keep CacheStorage out of the equation.

[login to view URL] = false to force local model loading.

The same failure repeats when directly importing pipeline in the DevTools console and invoking it with { device: 'wasm' }.

What a developer might want to inspect (no solutions implied)

The exact point in [login to view URL] where the WASM backend selection and ORT session creation occur (minified functions s, [login to view URL], T, ct.from_pretrained).

How Transformers v2.x is discovering and loading the ORT WASM binaries when wasmPrefix is provided and wasmPaths is absent.

Any assumptions the library makes about the shape/type of responses (e.g., ArrayBuffer vs Response/Blob) for the WASM files and for model files, and whether the custom VFS Response objects align with those expectations during the initialization pathway.

Differences in the initialization path between SIMD and BASIC modes that could trigger the reading 'buffer' TypeError in SIMD but a silent fallback in BASIC.

Whether backend hinting in v2.x expects a string vs array (observed both forms were tried), and how that affects the internal backend selection logic.

Any ORT/Transformers version coupling implicitly encoded in the bundle (e.g., compiled imports/exports signatures expected by the two wasm binaries currently deployed).

Current impact

The quiz app cannot generate questions/explanations because the text generation pipeline never initializes.

All other UI and model import flows behave as expected; failure is isolated to model inference startup in the browser." (client-provided description)