AI.js

Overview

Owns provider-selectable chat and the one-time caller-authority browser STT/TTS configuration, lifecycle, synthesis, transcription, and playback boundary.

• Artifact
  AI.js · esm
• Classification
  public first party
• Availability
  Browser + native bridge + TWiN Cloud
• Normalization
  Complete mutable caller-owned browser speech authority, SDK-owned provider registration/replacement/disposal, explicit STT/TTS activation, normalized Kokoro TTS execution with default capacity 4, explicit provider-neutral execution snapshots through status(role,{execution:true}), sticky readiness, canonical window.user readiness without compatibility-event object-identity admission or polling, exact ordered structural calls with required arguments.message, atomic nonblank all-ID tool-result sequencing, complete all-choice streaming/data validation, native Ollama structural adaptation, complete-response then all validated tool callbacks then completion ordering, observed async callbacks, Blob/File transcription, automatic repeated-formatting-mark removal from cloned outbound TTS input while caller and visible content stays exact, immediate exact-segment synthesis, per-call voice/speed capture, optional final-segment pause on the existing audio clock, optional per-call terminal playback completion, detached preparation with complete semantic DBOPFS audio reuse and same-owner pending request sharing, independent prepared playback without a model load for cached audio, ordered audio-clock scheduling, playable audio, active-generation TTS operation-failure routing, mute, and cancellation are normalized; provider/model/runtime/voice policy remains caller-owned.

Import and lifecycle

import * as module from '/arcane/modules/AI.js';

Browser import listens for user-entity-loaded and arcane-ollama-ready, installs window.ai once window.user is ready, emits ai-ready, and may perform provider configuration/transition, HTTP/native AI, microphone, and audio-playback side effects. User events prompt a readiness recheck without requiring their source detail to preserve window.user object identity; an immediate post-registration recheck catches an already-ready user without polling.

Application-facing behavior: Browser-speech protocol/event/error/reason constants; default AI; read-only providerRuntime, browserSpeechConfiguration, and browserSpeechDescriptor; explicit providerRuntime.status(role,{execution:true}) snapshots; streamTTS(text,end,options={}) with optional voice, speed, pauseAfterMs, and waitForPlayback; automatic speech-input cleanup with SDK-internal preparation metadata; prepareTTS({parts,storage,identity,signal,onState}) returning ordered segments, state, ready, getAudio(index), and cancel(); playPreparedTTS(prepared,{signal,onState}) returning state, error, finished, pause(), resume(), and stop() with one playback lane per AI and observational state/error callbacks; configure/dispose speech, route lifecycle, declaration-validated chat/stream requests with exact ordered calls, synthesis/transcription, and playback controls; initializes from current canonical user readiness, installs window.ai, and projects ai-ready plus active-generation ai-tts-failure.

Exports, signatures, parameters, and results

Parameter meanings and results

AI(llmService, sttService, ttsService, model, modelTTS, modelSTT) selects providers and models. configureProviders(config) updates routes without unloading them. transitionAI(llmService, sttService, ttsService, model, modelTTS, modelSTT) and transitionProviders(selections) stop queued audio, unload all three current roles, and then configure replacements; the first returns aggregate runtime status and the second returns the configured routes. startProviders({startMuted=true,startTranscription=false,signal=null}={}) declines the startup STT load unless explicitly requested without forcing an independently active STT role to unload, and setSpeechMuted(muted) owns TTS lifecycle. configureTTSSegmentation({punctuation,wordCadence}) selects sentence, any-punctuation, or no-punctuation boundaries plus an optional positive whole-word cadence without changing or discarding text; synthesis enters provider-capacity FIFO admission immediately and ready audio buffers play consecutively in original order on their owning AudioContext clocks. Every streamTTS(), finishTTS(), and fetchTTS() call automatically removes repeated same formatting marks from only a cloned outbound speech-input copy; caller, displayed, stored, and model text remains exact. The optional fetchTTS(payload,signal,preparation={}) preparation argument is SDK-internal, and {speechInputPrepared:true} prevents a second non-idempotent cleanup pass after an SDK owner has already prepared the copy; applications omit it. SDK 0.5.12 streamTTS(text,end,options={}) captures supplied voice and speed per extracted segment without changing instance defaults; omission uses current defaults. pauseAfterMs defaults to 0 and places finite nonnegative milliseconds after only the final extracted segment on the existing audio clock; invalid pauses throw RangeError. waitForPlayback defaults to false and retains the preparation boolean. With true, the boolean settles after every segment extracted by that invocation reaches a terminal state: true when all naturally end, false on terminal cancellation or failure. Options apply only to segments extracted in that invocation and are not retained with buffered end:false remainder; later calls supply their own options and finishTTS() uses defaults. No extracted segment means immediate true without waiting for earlier jobs; muted calls return false. Autoplay and recoverable resume waiting remain pending; a failed resume of a closed AudioContext terminates affected jobs with false playback results. A trailing pause delays the next queued audio, not the completed buffer promise. finishTTS() remains a preparation flush. stopAudio() cancels AI-owned streamed speech and prepared playback and settles pending playback promises false, while detached preparation continues. prepareTTS({parts,storage,identity,signal,onState}) returns {segments,state,ready,getAudio(index),cancel()} synchronously, retains exact source semantic inputs, applies speech formatting cleanup once before configured punctuation segmentation, and prepares without playback through the existing bounded provider queue. Optional storage {db,table,key} persists complete audio and MIME metadata in caller-owned DBOPFS and reuses matching semantic variants. Matching pending requests share synthesis on the same AI and storage group; cancelling one observer only cancels shared generation when no active observer remains. ready resolves after complete generation/reuse and optional durable saving, or rejects the complete error or AbortError; completed stored segments survive. onState synchronously receives {state,completed,total,segments,error}. playPreparedTTS(prepared,{signal,onState}) returns {state,error,finished,pause(),resume(),stop()} immediately, reuses the ordered audio-clock scheduler, and enables cached playback without loading the selected model; missing generation alone requests provider readiness. State/error getters and optional observational onState({state,error}) expose waiting, waiting-for-gesture, scheduled, paused, complete, stopped, or error; waiting-for-gesture identifies audio-unlock UI, while a false resume result alone is not a first-segment-ready signal. One AI has one playback lane: attaching prepared playback replaces preceding streamed/prepared audio, and streamTTS interrupts active prepared playback. finished is true after all audio naturally ends or false after stop, cancellation, or failure; final trailing silence does not delay completion, and failures use existing diagnostics/events. Playback controls and replacement do not cancel preparation, pause/resume affect only that playback audio context, and mute or provider replacement retain their broader provider lifecycle authority. Preparation cancellation prevents later synthesis; an existing shared provider load/unmute has no per-preparation signal and may finish. fetchSTT(audioFile,signal=null) returns the transcript and propagates the supplied cancellation signal. Selected TWiN Cloud TWIN LLM, OLLAMA LLM, and Core LOCAL_SPEACH STT/TTS routes expose truthful capability-only provider/2 readiness without probing or downloading. Built-in audio stays on device: saved OPENAI speech selections migrate to LOCAL_SPEACH / whisper-small for STT and LOCAL_SPEACH / kokoro for TTS, while non-local wrapper configurations reject. The TWiN access key is used only for remote LLM chat. Before importing AI.js or any module that imports it, hydrating a ready window.user, applying saved preferences, or starting providers, only exact uppercase OPENAI in tuple slot 0 (LLM provider) or slot 3 (default-model sentinel) changes to TWIN; all other values, openai-gpt-oss-120b, openai-gpt-oss-20b, OpenAI-compatible wire terminology, and separate Core provider:openai behavior remain unchanged; the SDK does not alias that identifier or rewrite saved preferences. TTS fills an omitted voice from the exact selected local model catalog defaultVoice; a saved OpenAI-route voice is not forwarded to Core or browser speech. Every active-generation non-abort TTS synthesis, decode, playback-start, or playback-resume failure emits ai-tts-failure with its complete Error and exact operation boundary; mute, cancellation, permission waiting, and superseded generations emit no failure and provider readiness is not rewritten. fetchRequest(options) preserves provider-native JSON. fetchRequest(options) and streamRequest(options) accept provider-neutral reasoningEffort values none, low, medium, high, and max; TWiN Cloud maps a supplied value to DigitalOcean reasoning_effort, omission preserves the provider default, and explicit openai-gpt-oss-120b or openai-gpt-oss-20b selection is preserved. streamRequest(options) forwards each complete provider chunk through onDataChunk, every choice's nonstructural content/reasoning through onChunk in provider order, the complete terminal response through onDataResult and onResponse, each complete validated structural call exactly once through onToolCall, and then awaits onComplete. Its application result is the ordered structural-call array for selected tool output, the complete completion object for multiple choices, or the ordinary single-result text/completion otherwise; partial structural deltas remain private. onRequest receives supplied transport metadata. Native Ollama calls are adapted before validation without inventing arguments.message. Browser speech normalizes shared Blob/File STT and WAV TTS requests at the provider boundary. Kokoro defaults to four Worker/model sessions; explicit execution capacity 1 through 4 remains supported. Fifth and later segments wait in the FIFO queue without dropping content, while playback schedules exact input order. Raising capacity trades memory for latency. Explicit providerRuntime.status(null,{execution:true}).roles.tts.execution reports provider-supplied requestedDevice, selectedDevice, maxConcurrentRequests, and activeRequestCount; selectedDevice is null while unloaded. Default status preserves sticky-state identity and never probes the provider. Execution reports do not establish physical GPU kernel overlap.

Events, side effects, and errors

Source-literal CustomEvent dispatches

No source-literal CustomEvent dispatch is part of this artifact.

Lifecycle and event flow

• consumes user-entity-loaded
• consumes arcane-ollama-ready
• emits ai-ready
• emits ai-tts-failure

Direct coded failures

• AI_REASONING_EFFORT_INVALID
• AI_NATIVE_LOCAL_REQUIRED
• AI_NATIVE_LOCAL_REQUIRED
• AI_PROVIDER_NOT_CONFIGURED
• AI_MODEL_INVALID
• AI_STT_DEVICE_ONLY
• AI_TTS_DEVICE_ONLY
• ARCANE_AI_PROVIDER_UNAVAILABLE
• ARCANE_AI_TTS_NATIVE_AUDIO_INVALID
• AI_ANDROID_NATIVE_SPEECH_UNAVAILABLE
• AI_REQUIRED_TOOL_UNAVAILABLE
• AI_TOOL_ARGUMENTS_INVALID
• AI_TOOL_ARGUMENTS_INVALID
• AI_STRUCTURED_OUTPUT_INVALID
• AI_REQUIRED_TOOL_CALL_MISSING
• AI_LOCAL_MODEL_REQUIRED
• AI_LOCAL_MODEL_REQUIRED
• ARCANE_AI_TTS_PROVIDER_AUDIO_INVALID
• ARCANE_AI_TTS_REQUEST_INVALID
• ARCANE_AI_TTS_REQUEST_INVALID
• ARCANE_AI_TTS_SIGNAL_INVALID
• ARCANE_AI_TTS_INPUT_INVALID
• ARCANE_AI_TTS_MODEL_INVALID
• ARCANE_AI_TTS_MODEL_REQUIRED
• ARCANE_AI_TTS_MODEL_SELECTION_MISMATCH
• ARCANE_AI_TTS_VOICE_INVALID
• ARCANE_AI_TTS_VOICE_REQUIRED
• ARCANE_AI_TTS_RESPONSE_FORMAT_INVALID
• ARCANE_AI_TTS_RESPONSE_FORMAT_INVALID
• ARCANE_AI_TTS_SPEED_INVALID
• ARCANE_AI_STT_SIGNAL_INVALID
• ARCANE_AI_STT_PROVIDER_TRANSCRIPT_INVALID

Exported Error subclasses

This artifact exports no Error subclass.

Documented failure behavior

• AI_NATIVE_LOCAL_REQUIRED
• AI_PROVIDER_NOT_CONFIGURED
• AI_MODEL_INVALID
• AI_LOCAL_MODEL_REQUIRED
• AI_REASONING_EFFORT_INVALID
• AI_STRUCTURED_OUTPUT_INVALID
• AI_REQUIRED_TOOL_UNAVAILABLE
• AI_REQUIRED_TOOL_CALL_MISSING
• AI_SERVICE_UNREACHABLE
• AI_REQUEST_FAILED
• AI_REQUEST_ABORTED
• AI_ANDROID_NATIVE_SPEECH_UNAVAILABLE
• AI_STT_DEVICE_ONLY
• AI_TTS_DEVICE_ONLY
• ARCANE_AI_MODEL_AUTHORITY_REQUIRED
• ARCANE_AI_PROVIDER_DISPOSED
• ARCANE_AI_PROVIDER_RUNTIME_INVALID
• ARCANE_AI_PROVIDER_UNAVAILABLE
• ARCANE_AI_REQUEST_ABORTED
• ARCANE_AI_ROLE_BUSY
• ARCANE_AI_ROLE_NOT_READY
• ARCANE_AI_INVALID_REQUEST
• ARCANE_AI_AUDIO_DECODE_UNAVAILABLE
• ARCANE_AI_AUDIO_DECODE_FAILED
• ARCANE_AI_INVALID_PROVIDER_RESULT
• ARCANE_AI_UNSUPPORTED_RESPONSE_FORMAT
• ARCANE_AI_TTS_HTTP_REQUEST_FAILED

Availability and capabilities

Browser + native bridge + TWiN Cloud. Complete mutable caller-owned browser speech authority, SDK-owned provider registration/replacement/disposal, explicit STT/TTS activation, normalized Kokoro TTS execution with default capacity 4, explicit provider-neutral execution snapshots through status(role,{execution:true}), sticky readiness, canonical window.user readiness without compatibility-event object-identity admission or polling, exact ordered structural calls with required arguments.message, atomic nonblank all-ID tool-result sequencing, complete all-choice streaming/data validation, native Ollama structural adaptation, complete-response then all validated tool callbacks then completion ordering, observed async callbacks, Blob/File transcription, automatic repeated-formatting-mark removal from cloned outbound TTS input while caller and visible content stays exact, immediate exact-segment synthesis, per-call voice/speed capture, optional final-segment pause on the existing audio clock, optional per-call terminal playback completion, detached preparation with complete semantic DBOPFS audio reuse and same-owner pending request sharing, independent prepared playback without a model load for cached audio, ordered audio-clock scheduling, playable audio, active-generation TTS operation-failure routing, mute, and cancellation are normalized; provider/model/runtime/voice policy remains caller-owned.

Normalizes explicitly selected provider/2 LLM, on-device Whisper STT, and on-device Kokoro TTS routes alongside configured TWiN Cloud HTTPS, Arcane.ollama.chat, and Arcane.speech paths. Each route keeps its real browser, native/Core, or cloud availability and never silently falls back.

Contract example

import AI from '/arcane/modules/AI.js';

// The application supplies this runtime argument; never put its key in source.
async function sayHello(applicationRuntime) {
    const ai = new AI();
    ai.twinKey = applicationRuntime.twinKey;
    try {
        const response = await ai.fetchRequest(
            {
                messages: [{role: 'user', content: 'Hello!'}]
            }
        );
        console.log(JSON.stringify(response, null, 2));
    } catch (error) {
        console.error(error.code, error.message);
    }
}

Related reference

• Module index
• Normalized AI decision guide
• Arcane.ai.chat()
• Arcane.runtime.current()
• Arcane.ai namespace