Skip to content
aparté

Transformers.js (local)

Run models 100% in the browser via Transformers.js (WebGPU, with a WASM fallback) — no API, no key, no server. Inference runs off the main thread in a Web Worker; weights download once and persist in the Cache API.

Terminal window
npm install @aparte/provider-transformers @huggingface/transformers

@huggingface/transformers is a peer dependency — it’s heavy and ships its own onnxruntime, so you bring the version you want.

import { AparteConfig, DirectTransport } from '@aparte/core';
import { TransformersProvider, registerModel } from '@aparte/provider-transformers';


registerModel({
  id: 'onnx-community/Qwen2.5-0.5B-Instruct',
  name: 'Qwen2.5 0.5B',
  task: 'text-generation',
  capabilities: ['streaming'],
  dtype: 'q4',
});
AparteConfig.registerAIProvider(TransformersProvider);
AparteConfig.setTransport(new DirectTransport({ byok: true }));

The provider owns its I/O (it runs inference locally), so DirectTransport just delegates to it.

Managing downloads & cache

Section titled “Managing downloads & cache”

Downloading and status are methods on the provider you registered:

  • TransformersProvider.prepareModel(modelId, onProgress) — download + load a model, reporting progress.
  • TransformersProvider.getModelStatus(modelId) — 'ready' | 'cached' | 'not-downloaded'.

Cache and hardware are standalone helpers — import them from @aparte/provider-transformers:

  • listCachedModels() / deleteCachedModel(modelId) — inspect and clear the on-disk cache.
  • setMaxCachedModels(n) — cap how many models are kept (oldest evicted; 0 = unlimited).
  • detectHardware() / setComputeDevice('auto' | 'webgpu' | 'wasm') — pick a device / default model by tier. Call setHardwareTierModels({ low, mid?, high }) first — otherwise detectHardware()’s recommendedModelId is always ''.