Browser automation runtime, context providers, and extension host contracts for AIPex
npm install @aipexstudio/browser-runtimeChrome/Chromium runtime implementations for @aipexstudio/aipex-core.
This package is where browser-specific code lives (Manifest V3 friendly). It provides:
- Browser tools (allBrowserTools) for tab/page interaction
- Context providers for common browser data sources (tabs, bookmarks, history, current page, screenshots)
- Storage adapters for extension environments (ChromeStorageAdapter, IndexedDBStorage)
- A CDP-based automation layer (via chrome.debugger) and related helpers
- Runtime contracts used by the AIPex extension (hosts, addons, omni action registry)
> Note: These APIs depend on chrome.* and/or indexedDB. They are not meant to run in plain Node.js.
AIPex is split into layers so each stays focused:
- @aipexstudio/aipex-core: platform-agnostic agent + events + contexts + sessions
- @aipexstudio/browser-runtime: Chrome/extension implementations (tools, providers, storage, automation)
- @aipexstudio/aipex-react: React UI toolkit that depends only on core
- browser-ext: the actual extension that wires everything together
allBrowserTools is a curated bundle of FunctionTools that an agent can call.
It includes:
- Tab management: list/switch/open/duplicate/close, basic grouping helpers
- UI operations: locate elements, click, hover, fill inputs/forms, wait
- Page content: metadata, scrolling, highlighting
- Screenshots: capture to data URL / clipboard
- Downloads: save text/images from the agent workflow
- Human-in-the-loop interventions: request/cancel interventions
Tool names included (strings used for tool-calling):
- Tabs: get_all_tabs, get_current_tab, switch_to_tab, create_new_tab, get_tab_info, duplicate_tab, close_tab, organize_tabs, ungroup_tabs
- UI ops: search_elements, click, fill_element_by_uid, get_editor_value, fill_form, hover_element_by_uid, wait
- Page: get_page_metadata, scroll_to_element, highlight_element, highlight_text_inline
- Screenshot: capture_screenshot, capture_tab_screenshot, capture_screenshot_to_clipboard
- Download: download_text_as_markdown, download_image, download_chat_images, download_current_chat_images
- Interventions: list_interventions, get_intervention_info, request_intervention, cancel_intervention
> Note: take_snapshot exists but is intentionally not included in allBrowserTools because it is used internally.
This package includes providers and a convenience registration helper:
- allBrowserProviders
- registerDefaultBrowserContextProviders(manager)
Providers included by default:
- CurrentPageProvider
- TabsProvider
- BookmarksProvider
- ScreenshotProvider
- HistoryProvider
- ChromeStorageAdapter: implements KeyValueStorage using chrome.storage.
- Supports "local" and "sync" areas
- Falls back to localStorage when chrome.storage is unavailable (useful for non-extension dev)
- IndexedDBStorage: a small KeyValueStorage implementation backed by IndexedDB
Under automation/, you’ll find building blocks for CDP-based workflows (via chrome.debugger):
- DebuggerManager
- SnapshotManager
- SmartLocator / SmartElementHandle
- Snapshot text search utilities (searchSnapshotText, parseSearchQuery, hasGlobPatterns)
``bash`
npm install @aipexstudio/browser-runtimeor
pnpm add @aipexstudio/browser-runtime
Peer dependencies:
- @types/chrome (for TypeScript)react
- (optional; only needed if you use React-related helpers)
`ts
import { google } from "@ai-sdk/google";
import { AIPex, aisdk } from "@aipexstudio/aipex-core";
import { allBrowserTools } from "@aipexstudio/browser-runtime";
const agent = AIPex.create({
instructions: "You can control the current browser tab.",
model: aisdk(google("gemini-2.5-flash")),
tools: allBrowserTools,
});
`
`ts
import { ContextManager } from "@aipexstudio/aipex-core";
import { registerDefaultBrowserContextProviders } from "@aipexstudio/browser-runtime";
const contextManager = new ContextManager({ autoInitialize: true });
registerDefaultBrowserContextProviders(contextManager);
`
@aipexstudio/aipex-react accepts any KeyValueStorage implementation for persisting settings.chromeStorageAdapter
In a Chrome extension, you can pass :
`tsx
import { Chatbot } from "@aipexstudio/aipex-react";
import { chromeStorageAdapter } from "@aipexstudio/browser-runtime";
export function App({ agent }: { agent: any }) {
return
}
`
`ts
import { executeScriptInActiveTab } from "@aipexstudio/browser-runtime";
const title = await executeScriptInActiveTab(() => document.title, []);
console.log(title);
`
`ts
import { IndexedDBStorage } from "@aipexstudio/browser-runtime";
const storage = new IndexedDBStorage<{ id: string; value: string }>({
dbName: "aipex",
storeName: "sessions",
});
`
- allBrowserTools: FunctionTool[]interventionTools: FunctionTool[]
- registerDefaultBrowserTools(registryLike)
- getActiveTab(): Promise
- executeScriptInTab(tabId, func, args)
- executeScriptInActiveTab(func, args)
-
- allBrowserProviders: ContextProvider[]registerDefaultBrowserContextProviders(manager)
- CurrentPageProvider
- Provider classes: , TabsProvider, BookmarksProvider, ScreenshotProvider, HistoryProvider
- ChromeStorageAdapterchromeStorageAdapter: ChromeStorageAdapter
- IndexedDBStorage
- IndexedDBConfig
-
- DebuggerManager, debuggerManagerSnapshotManager
- , snapshotManagerSmartLocator
- , SmartElementHandlesearchSnapshotText
- , parseSearchQuery, hasGlobPatterns
- RuntimeAddonNoopBrowserAutomationHost
- InMemoryOmniActionRegistry
- NullInterventionHost
- NoopContextProvider
-
From the repository root:
`bash``
pnpm --filter @aipexstudio/browser-runtime build
pnpm --filter @aipexstudio/browser-runtime typecheck
MIT