**Bringing the power of Large Language Models (LLMs) directly to your Angular applications with WebLLM.**
npm install ngx-web-llmngx-web-llm is an Angular library that wraps the WebLLM JavaScript library, allowing you to run LLM inference entirely in the browser. This enables privacy-preserving, offline-capable AI features with no server-side dependencies for inference.
0.0.1
WebLlmService.
bash
npm install ngx-web-llm @mlc-ai/web-llm
or
yarn add ngx-web-llm @mlc-ai/web-llm
`
Basic Usage
1. Import and Configure the Module:
Import NgxWebLlmModule into your Angular application. For standalone applications, use importProvidersFrom. For NgModule-based applications, import into your desired module.
For Standalone Applications (app.config.ts):
`typescript
import { ApplicationConfig, importProvidersFrom } from "@angular/core";
import { provideRouter } from "@angular/router";
import { NgxWebLlmModule } from "ngx-web-llm"; // Ensure this path is correct
import { routes } from "./app.routes";
export const appConfig: ApplicationConfig = {
providers: [
provideRouter(routes),
importProvidersFrom(
NgxWebLlmModule.forRoot({
// Optional: Configure default model, preloading, caching etc.
defaultModelId: "Llama-3.1-8B-Instruct-q4f32_1-MLC", // Example
preload: true,
// workerScriptUrl: 'path/to/your/web-llm.worker.js', // Only if using Web Worker
})
),
],
};
`
For NgModule-based Applications (app.module.ts or feature module):
`typescript
import { NgModule } from "@angular/core";
import { BrowserModule } from "@angular/platform-browser";
import { NgxWebLlmModule } from "ngx-web-llm"; // Ensure this path is correct
import { AppComponent } from "./app.component";
@NgModule({
declarations: [AppComponent],
imports: [
BrowserModule,
NgxWebLlmModule.forRoot({
// Optional: Configure default model, preloading, caching etc.
defaultModelId: "Llama-3.1-8B-Instruct-q4f32_1-MLC", // Example
preload: true,
// workerScriptUrl: 'path/to/your/web-llm.worker.js', // Only if using Web Worker
}),
],
providers: [],
bootstrap: [AppComponent],
})
export class AppModule {}
`
2. Inject and Use WebLlmService:
`typescript
import { Component, OnInit } from "@angular/core";
import { FormsModule } from "@angular/forms"; // Needed for ngModel
import { CommonModule } from "@angular/common"; // Needed for async pipe and new control flow
import { WebLlmService, NgxMessage } from "ngx-web-llm"; // Ensure path is correct
@Component({
selector: "app-chat",
standalone: true, // Assuming standalone component for modern Angular
imports: [CommonModule, FormsModule], // Import necessary modules
template:
,
})
export class ChatComponent implements OnInit {
messages: NgxMessage[] = [];
userInput: string = "";
isGenerating: boolean = false;
constructor(public webLlm: WebLlmService) {}
async ngOnInit() {
// Example: Explicit initialization if not relying on preload
// if (!this.webLlm.isEngineReady && !(await (this.webLlm.isLoading$ | async))) { // Check isLoading correctly
// try {
// // Initialize with the default model (or specify one)
// // Set useWebWorker to false if not using a worker, or true if workerScriptUrl is configured
// await this.webLlm.initializeEngine(undefined, undefined, false);
// console.log('Engine initialized successfully!');
// } catch (e) {
// console.error('Engine initialization failed:', e);
// }
// }
}
sendMessage(): void {
if (!this.userInput.trim() || !this.webLlm.isEngineReady || this.isGenerating) return;
const userMessage: NgxMessage = { role: "user", content: this.userInput };
this.messages = [...this.messages, userMessage];
const currentInput = this.userInput;
this.userInput = "";
this.isGenerating = true;
let assistantResponse = "";
const assistantMessagePlaceholder: NgxMessage = { role: "assistant", content: "..." };
this.messages = [...this.messages, assistantMessagePlaceholder];
this.webLlm.streamChatDeltas({ messages: [{ role: "user", content: currentInput }] }).subscribe({
next: (deltaContent) => {
assistantResponse += deltaContent;
this.messages = this.messages.map((msg, index) => (index === this.messages.length - 1 ? { ...msg, content: assistantResponse } : msg));
},
error: (err) => {
console.error("Error during chat streaming:", err);
this.messages = this.messages.map((msg, index) => (index === this.messages.length - 1 ? { ...msg, content: "Error generating response." } : msg));
this.isGenerating = false;
},
complete: () => {
this.isGenerating = false;
},
});
}
}
`
Configuration (
NgxWebLlmConfig)
When calling NgxWebLlmModule.forRoot(config), you can provide an NgxWebLlmConfig object:
`typescript
export interface NgxWebLlmConfig {
defaultModelId?: string; // e.g., 'Llama-3.1-8B-Instruct-q4f32_1-MLC'
defaultEngineConfig?: webllm.MLCEngineConfig; // Advanced engine settings from WebLLM
preload?: boolean; // If true, initializes with defaultModelId on service creation
workerScriptUrl?: string; // Optional: Path to web-llm.worker.js if choosing to use Web Worker.
// The application is responsible for serving this worker script.
caching?: {
usePersistentStorage?: boolean; // Request persistent storage for models
storageQuotaBytes?: number; // (Currently informational) Desired quota
};
}
`
WebLlmService API Highlights
- Properties & Observables:
- currentModelId$: Observable
- engineReady$: Observable
- initializationProgress$: Observable
- isLoading$: Observable
- get prebuiltAppConfig(): webllm.AppConfig
- Core Methods:
- async initializeEngine(modelId?: string, config?: webllm.MLCEngineConfig, useWebWorker?: boolean): Promise: Initializes the WebLLM engine. useWebWorker is false by default; set to true and provide workerScriptUrl in config if using a worker.
- chatCompletions(request: NgxChatCompletionRequest): Observable
- streamChatDeltas(request: Omit
- async reloadModel(modelId: string, chatOpts?: webllm.ChatOptions): Promise
- async unloadEngine(): Promise
- getAvailableModels(): NgxModelRecord[]
- async interruptGeneration(): Promise
- async getRuntimeStats(): Promise
- async resetChat(keepSystemPrompt?: boolean): Promise
Troubleshooting
- Ensure @mlc-ai/web-llm` is correctly installed as a peer dependency.