Local runtime APIs

Runs the hand-written WebGPU compute kernels from the webml-community HF Spaces (LFM2.5, Gemma 4) under Bun's headless WebGPU (Metal on macOS) or in the browser. Same kernels, unmodified — just no server in front of them.

bun add @valora-ai/ai-sdk @valora-ai/react @valora-ai/gemma
# Add provider packages as you use them:
bun add @valora-ai/lfm2 @valora-ai/kokoro @valora-ai/moonshine-stt @valora-ai/silero-vad

Models

Speech: Kokoro-82M (TTS, via transformers.js) and Whisper / LFM2.5-Audio (STT). VAD is Silero.

Valora keeps model families in separate provider packages. Install only the providers your app imports.

Requires

• Bun ≥ 1.3.14 for the repo CLI.
• A WebGPU-capable GPU.

CLI

Run the WebGPU kernels under Bun's headless WebGPU:

bun run list                 # list models
bun run chat                 # pick a model, chat in terminal
bun run chat lfm2.5-350m     # chat with a specific model
bun run serve lfm2.5-350m    # OpenAI-compatible endpoint on :8080

Point any OpenAI client at http://localhost:8080/v1. Gated/private HF repos: set HF_TOKEN.

AI SDK provider (in-process, no server)

A native AI SDK 7 (LanguageModelV3) provider runs the engines directly — no HTTP hop, even locally:

import { generateText, streamText } from 'ai';
import { valora } from '@valora-ai/ai-sdk';

const { text } = await generateText({ model: valora('gemma-4-e2b'), prompt: 'hi' });

const result = streamText({ model: valora('lfm2.5-350m'), prompt: 'Count to 5.' });
for await (const chunk of result.textStream) console.log(chunk);

valora('id') is the default registry path for built-in model ids. For browser apps that need app-owned ids, private/gated Hugging Face repos, or one model configuration shared with voice, create a configured provider and adapt it into AI SDK:

import { generateText } from 'ai';
import { createValora } from '@valora-ai/ai-sdk';
import { createLfm2Provider } from '@valora-ai/lfm2/provider';

const lfm2 = createLfm2Provider({
  auth: { type: 'bearer', token: hfToken },
  models: {
    'team-chat': {
      source: {
        type: 'huggingface',
        repo: 'LiquidAI/LFM2.5-350M-GGUF',
        revision: 'main',
        file: 'LFM2.5-350M-Q4_0.gguf',
      },
    },
  },
});

const configuredValora = createValora({ languageModels: [lfm2] });
await generateText({ model: configuredValora('team-chat'), prompt: 'hi' });

The same lfm2 object exposes native Valora engines through lfm2.languageModel('team-chat'); @valora-ai/voice consumes that directly. Bearer auth belongs in provider/load configuration. accessToken still works as a compatibility alias, but secrets should not be stored on ModelCard.meta.

The same package exposes explicit audio adapters:

import {
  createValoraSpeechModel,
  createValoraTranscriptionModel,
} from '@valora-ai/ai-sdk';

const speech = createValoraSpeechModel('kokoro', async () => speechEngine);
const transcription = createValoraTranscriptionModel('whisper', async () => sttEngine);

Speech returns WAV bytes. Transcription accepts mono 16-bit PCM WAV in this first pass, decodes it to Float32Array, and calls the local TranscribeEngine.

Model file source & auth

Every vendor provider factory (createLfm2Provider, createGemmaProvider, createWhisperProvider, …) takes a hub option that points model downloads at a mirror/caching proxy and configures auth, independent of the model catalogue:

createLfm2({
  hub: {
    endpoint: 'https://my-proxy.workers.dev/hf',
    auth: process.env.HF_TOKEN, // bearer-token shorthand; false forces anonymous
  },
});

endpoint/fetch resolve with this precedence (highest first):

1. Explicit call-time load options (e.g. provider.load(card, opts)).
2. A per-model source.endpoint override on the ModelCard.
3. ProviderOptions.hub.
4. ProviderOptions.loadOptions.

If no auth token resolves through any of those and hub.auth was not explicitly set to false, Node falls back to process.env.HF_TOKEN — CLI and server usage works without wiring auth by hand. Pass hub: { auth: false } to force anonymous downloads even when HF_TOKEN is set.

Browser apps that want every Hugging-Face-prefixed fetch rewritten — not just the ones going through a configured provider, but the vendored WebGPU kernels too, which call global fetch directly — should use installFetchRewrite from @valora-ai/runtime-utils as a blanket, app-level rewrite instead of configuring hub per provider:

import { installFetchRewrite } from '@valora-ai/runtime-utils';

installFetchRewrite({ to: 'https://my-proxy.workers.dev/hf' });

See examples/hf-proxy for a full caching-proxy Worker and the fetch-shim pattern it pairs with.

Silero VAD assets (vad.worklet.bundle.min.js, the ONNX model, onnxruntime-web WASM) are fetched from their own public CDN via assets: { baseAssetPath, onnxWASMBasePath } on createSileroVad — a separate, unauthenticated source that is not a Hugging Face hub file. hub.endpoint does not apply to VAD assets.

React hooks (in-browser, WebGPU)

import { createGemmaProvider } from '@valora-ai/gemma/provider';
import {
  createValoraRuntime,
  ValoraRuntimeProvider,
  useLocalModel,
  useLocalRealtime,
  useLocalTTS,
  useTranscribe,
  useVAD,
} from '@valora-ai/react/local';

const runtime = createValoraRuntime({
  languageModels: [createGemmaProvider()],
});

function Voice() {
  const { status, send } = useLocalModel('gemma-4-e2b');
  const { speak } = useLocalTTS();
  const { transcribe } = useTranscribe();
  useVAD({ onSpeechEnd: (audio) => transcribe(audio) }); // Float32 @ 16kHz
}

export default function App() {
  return (
    <ValoraRuntimeProvider runtime={runtime}>
      <Voice />
    </ValoraRuntimeProvider>
  );
}

useLocalModel('gemma-4-e2b') can still run through the legacy static browser loader when no runtime provider is mounted. Mount a ValoraRuntimeProvider when the app needs configured model sources, bearer auth, custom ids, or one provider setup shared with local voice. Hooks need a WebGPU browser and a bundler (Vite / Next client component). These four hooks are the same native engine contracts @valora-ai/voice's createVoiceAgent expects.

useLocalRealtime(agent) is the hook for an assembled VoiceAgent: it returns the session snapshot plus connect, disconnect, sendText, interrupt, mute, and unlock, while keeping capture and inference local.

See the local API reference.