Creates a new ExecuTorch LLM instance.
Parameters for the instance.
Path or URL of the LLM model (.pte).
Path or URL of the tokenizer (tokenizer.json).
Path or URL of the tokenizer config (tokenizer_config.json).
Download progress callback (0-1).
Generation configuration forwarded to ExecuTorch.
System prompt used when the history has none. Empty string disables it.
Generates a completion from a list of messages, streaming tokens to callback.
Conversation history for the model.
Token-level streaming callback.
Promise that resolves to the full generated string.
Interrupts the current generation and resolves once it has settled. The pending generate promise resolves with the tokens produced so far. Does nothing when no generation is running.
Downloads (if needed) and loads the model, tokenizer and chat template via react-native-executorch.
Overlapping calls share one load. An unload issued while the load
is in flight wins unless load is called again before it settles.
Promise that resolves to the same instance.
Unloads the underlying model and releases its native resources. A load still in flight is awaited first so that nothing is left behind.
ExecuTorch-based implementation of LLM for React Native.
Each generate call is stateless: the full message history is rendered through the model's chat template and fed to the runner from a fresh KV cache. When the history does not fit the model's context window, the oldest turns are dropped. The model itself is loaded once in load and kept in memory.