Class ExecuTorchLLM

ExecuTorch-based implementation of LLM for React Native.

Implements

LLM

Index

Constructors

Methods

generate interrupt load unload

Constructors

constructor

new ExecuTorchLLM(params: ExecuTorchLLMParams): ExecuTorchLLM
Creates a new ExecuTorch LLM instance.
Parameters
- params: ExecuTorchLLMParams
  Parameters for the instance.
  - modelSource
    Source of the LLM model.
  - tokenizerSource
    Source of the tokenizer.
  - tokenizerConfigSource
    Source of the tokenizer config.
  - onDownloadProgress
    Download progress callback (0-1).
  - responseCallback
    Callback invoked with final full response string.
  - chatConfig
    Chat configuration forwarded to ExecuTorch.
Returns ExecuTorchLLM

Methods

generate

generate(
messages: Message[],
callback: (token: string) => void,
): Promise<string>
Generates a completion from a list of messages, streaming tokens to callback.
Parameters
- messages: Message[]
  Conversation history for the model.
- callback: (token: string) => void
  Token-level streaming callback.
Returns Promise<string>
Promise that resolves to the full generated string.

Implementation of LLM.generate

interrupt

interrupt(): Promise<void>
Interrupts current generation. Note: current ExecuTorch interrupt is synchronous. Awaiting this method will not guarantee completion.

Returns Promise<void>
Implementation of LLM.interrupt

load

load(): Promise<ExecuTorchLLM>
Loads the model and config via react-native-executorch, and applies configuration.

Returns Promise<ExecuTorchLLM>
Promise that resolves to the same instance.

Implementation of LLM.load

unload

unload(): Promise<void>
Unloads the underlying module. Note: current ExecuTorch unload is synchronous. Awaiting this method will not guarantee completion.

Returns Promise<void>
Implementation of LLM.unload